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orientation and/or position of the taiget from 
those lines (120, 122). 
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5 MACHINE VISION CALIBRATION TARGETS AND METHODS 

OF DETERMINING THEIR LOCATION AND ORIENTATION IN AN IMAGE 

Reservation of Copyright 

10 The disclosure of this patent document contains material which is subject to 

copyright protection. The owner thereof has no objection to facsimile reproduction by 
anyone of the patent document or the patent disclosure, as it appears in the U.S. Patent and 
Trademark Office patent file or records, but otherwise reserves all copyright rights 
whatsoever. 

15 

Background of the Invention 

The invention pertains to machine vision and, more particularly, to calibration 
targets and methods for determining their location and orientation in an image. 

20 

Machine vision refers to the automated analysis of an image to determine 
characteristics of objects and other features shown in the image. It is often employed in 
automated manufacturing lines, where images of components are analyzed to determine 
placement and alignment prior to assembly. Machine vision is also used for quality 
25 assurance. For example, in the pharmaceutical and food packing industries, images of 

packages are analyzed to insure that product labels, lot numbers, "freshness" dates, and the 
like, are properly positioned and legible. 

In many machine vision applications, it is essential that an object whose image is 
30 to be analyzed include a . calibration target. Often a cross-shaped symbol, the target 
facilitates determining the orientation and position of the object with respect to other 
features in the image. It also facilitates correlating coordinate positions in the image with 
those in the "real worid," e.g., coordinate positions of a motion stage or conveyor bek on 
which the object is placed. A calibration target can also be used to facilitate determining 
35 the position and orientation of the camera with respect to the real world, as well as to 
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facilitate determining the camera and lens parameters such as pixel size and lens 
distortion. 

In addition to cross-shaped marks, the prior art suggests the use of arrays of dots, 
5 bulls-eyes of concentric circles, and parallel stripes as calibration targets. Many of these 
targets have characteristics that make difficult finding their centers and orientations. This 
typically results fi-om lack of clarity when the targets and, particularly, their borders are 
imaged. It also results from discrepancies in conventional machine vision techniques used 
to analyze such images. For example, the edges of a cross-shaped target may be 
10 imprecisely defined in an image, leading a machine vision analysis system to v^ongly 

interpret the location of those edges and, hence, to misjudge the mark's center by a fi-action 
of a pixel or more. By way of further example, a localized defect in a camera lens may 
cause a circular calibration mark to appear as an oval, thereby, causing the system to 
misjudge the image's true aspect ratio. 

15 

In addition to the foregoing, many of the prior art calibration targets are useful 
only at a limited range of magnifications. Parallel stripes, for example, do not provide 
sufficient calibration information unless many of them appear in an image. To 
accommodate this, a machine vision system must utilize lower magnification. However, 
20 as the magnification decreases, so does the ability of the machine vision equipment to 

distinguish between individual stripes. Similar drawbacks limit the usefulness of the other 
prior art calibration targets for use in all but a narrow range of magnifications. 

Though the art suggests the use of checkerboard patterns as alignment marks, the 
25 manner in which images of those marks are analyzed by conventional machine systems 
also limits their utility to a limited range of magnifications. Particularly, prior art systems 
obtain alignment information from checkerboard marks by identifying and checking their 
comers, e.g., the eight black (or white) comers in a black-and-white image. By relying on 
comers, the systems necessitate that images show entire checkerboards, yet, with 
30 sufficient resolution to insure accurate detection and analysis. 
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An object of this invention is to provide an improved calibration targets and 
methods for machine vision analysis thereof 



A related object is to provide calibration targets and analysis methods reliable at a 
5 wide range of magnifications. 



A further object is to provide such methods as can be readily implemented on 
conventional digital data processors or other conventional machine vision analysis 
equipment. 

Yet still another object of the invention is to provide such methods that can rapidly 
analyze images of calibration target without undue consumption of resources. 
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Summary of the Invention 

The foregoing objects are among those attained by the invention, which provides 
in one aspect a machine vision method for analysis of a calibration target of the type 
5 having two or more regions, each having a different "imageable characteristic" (e.g., a 
diflferent color, contrast, or brightness) from its neighboring region(s). Each region has at 
least two edges - referred to as "adjoining edges" - that are linear and that are direaed 
toward and, optionally meet at, a reference point (e.g., the center of the target or some 
other location of interest). The method includes generating an image of the target, 
10 identifying in the image features corresponding to the adjoining edges, and determining 
the orientation and/or position of the target from those edges. 

In another aspect, the invention provides a method as described above for 
analyzing a target of the type that includes four regions, where the adjoining edges of 
15 each region are perpendicular to one another, and in which each region in the target has a 
different imageable characteristic from its edge-wise neighbor. The edges of those regions 
can meet, for example, at the center of the target, as in the case of a four-square 
checkerboard. 

In yet another aspect, the invention provides a method as described above for 
determining an orientation of the target as a function of the angle of the edges identified in 
the image and for determining the location of the reference point as an intersection of lines 
fitted to those edges. In regard to the former, the invention provides a method of 
determining the orientation of a target in an image by applying a Sobel edge tool to the 
image to generate a Sobel angle image, and by generating a angle histogram from that 
angle image. In an alternate embodiment, the orientation is determined by applying a 
Hough line tool to the image and determining the predominant angle of the edges 
identified by that tooL 

30 In regard to the location of the reference point, one aspect of the invention calls for 

locating the adjoining edges by applying a cahper vision tool to the image, beginning at an 
approximate location of the reference point. That approximate location of the reference 
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point can itself be determined by applying a Hough line vision tool to the image m order 
to find lines approximating the adjoining edges and by determining an intersection of 
those lines. Alternatively, the approximate location of the reference point can be 
determined by performing a binary or grey scale correlation to find where a template 
5 representing the edges most closely matches the image. 

In another alternate embodiment, the approximate location of the reference point is 
determined by applying a projection vision tool to the image along each of the axes with 
which the adjoining edges align. A first difference operator vision tool and a peak 
10 detector vision tool are applied to the output of the projection tool (i.e., to the projection) 
in order to find the approximate location of the edges. 



The invention has wide application in industry and research applications. It 
facilitates the calibration of images by permitting accurate determination of target location 
15 and orientation, regardless of magnification. Thus, for example, an object bearing a target 
can be imaged by multiple cameras during the assembly process, with accurate 
determinations of location and orientation made from each such image. 

These and other aspects of the invention are evident in the drawings and in the 
20 description that follows. 
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Brief Description of the Drawings 



A more complete understanding of the invention may be attained by reference to 
the drawings, in which: 

5 

Figure 1 A - IC depict calibration targets according to the invention; 

Figure ID depicts the effect of rotation on the target depicted Figure IB; 

10 Figure 2 depicts an object according to the invention incorporating a calibration 

target of the type depicted in Figure IB; 

Figure 3 depicts a machine vision system according to the invention for 
determining the reference point and orientation of a calibration target; 

15 

Figures 4 and 5 depict a method according to the invention for interpreting an 
image of a calibration target to determine a reference point and orientation thereof; and 

Figure 6 illustrates the magnification invariance of a target according to the 
20 invention. 
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Detailed Description of the Dlustrated Embodiment 

Figures 1 A - IC depict calibration targets according to the invention. Referring to 
Figure 1 A, there is shown a target 10 according to the invention having three regions 12, 
5 14, 16. Each region is bounded by at least two linear edges that are oriented toward a 
reference location or reference point 18 on the target. Thus, for example, region 12 is 
bounded by edges 20, 24; region 14 is bounded by edges 20, 22; and region 16 is bounded 
by edges 22, 24. As evident in the drawings, the edges are shared by adjoining regions 
and, hence, are referred to below as "adjoining edges." Thus, region 12 shares edge 20 
10 with region 14; region 14 shares edge 22 with region 16; and region 16 shares edge 24 
with region 12. In the illustration, the reference point 14 is at the center of target 10, 
though, those skilled in the art will appreciate that the reference point can be positioned 
elsewhere. 

15 Each of the regions has a different imageable characteristic from its neighboring 

regions. As used herein, an "imageable characteristic" is a characteristic of a region as 
imaged by a machine vision system (e.g., of the type shown in Figure 3) and, particularly, 
as imaged by an image capture device used by such a system. For example, in the 
illustration, region 12 has the characteristic of being colored black; region 14, white; and 

20 region 16, gray. In addidon to color, imageable characteristics useful with conventional 
machine vision systems - which typically uulize image capture devices operational in 
visual spectrum - include contrast, brightness, and stippling. 

Those skilled in the art will appreciate that any other characteristics by which a 
25 region may be identified and distinguished in an image are suitable for practice of the 
invention. Thus, for example, for a machine vision system that utilizes a temperature- 
sensitive (or infrared) image capture device, an imageable characteristic is temperature. 
By way of further example, for a machine vision system that utilizes a nuclear decay 
radiation-sensitive image capture device, an imageable characteristic is emitted radiation 
30 intensity or frequency. 
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As shown in the illustration, the adjoining edges 20, 22, 24 compnse straight linear 
segments. Those edges are implicitly defined as the borders between regions that, 
themselves, have different imageable characteristics. Thus, for example, edge 20 is a 
straight linear segment defined by the border between black region 12 and white region 
5 14. Likewise, edge 24 is defined by the border between black region 12 and gray region 
16. Further, edge 22 is defined by the border between white region 14 and grey region 16. 

Figure IB depicts a calibration target 30 according to the invention having four 
rectangular (and, more particularly, square) regions 32, 34, 36, 38, As above, each region 

10 is bounded by at least two linear edges that are oriented toward a reference point 40 at the 
center of the target. Thus, for example, region 32 is bounded by edges 42, 44; region 34 is 
bounded by edges 42, 46; and so forth. As above, these edges are shared by adjoining 
regions. Thus, region 32 shares edge 42 with region 34, and so forth. Each region in 
target 30 has a different imageable characteristic from its edge- wise neighbor. Hence, 

15 regions 32 and 36 are white, while their edge-wise adjoining neighbors 34, 38 are black. 

Figure IC depicts a calibration target 50 according to the invention having five 
regions 52, 54, 56, 58, 60, each having two linear edges directed toward a reference point 
62. The adjoining regions are of differing contrast, thereby, defining edges at their 
20 common borders, as illustrated. Although the edges separating the regions 52 - 60 of 

target 50 are directed toward the reference point 62, they do not meet at that location. As 
evident in Figure IC, no marker or other element imageable characteristic is provided at 
reference point 62. 

25 Those skilled in the art will appreciate that, in addition to the calibration targets 

shown in Figures lA - IC, targets with still more regions (or as few as two regions) and 
shapes, otherwise in accord with the teachings hereof, fall within the scope of the 
invention. Moreover, it will be appreciated that targets may be of any size and that their 
regions need not be of uniform size. Still further, it will be appreciated that the outer 

30 borders of the targets need not be linear and may, indeed, take on any shape. 
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Figure 2 depicts an object according to the invention for use m macnine vision 
imaging, detection, and/or manipulation having a calibration target according to the 
invention coupled thereto. In the illustration, the object is an integrated circuit chip 70 
having coupled to the casing thereof a calibration target 72 of the type shown in Figure 

5 IB. Other targets according to the invention, of course, can likewise be coupled to the 
object 70. The targets can be coupled to the object by any known means. For example, 
they can be molded onto, etched into, or printed on the surface of the object. By way of 
further example, decals embodying the targets can be glued, screwed or otherwise affixed 
to the object. Moreover, by way of still further example, calibration plates incorporating 

10 the targets can be placed on the object and held in place by friction. In addition to 

integrated circuit chips, the object can include any other objects to which a target can be 
coupled, such as printed circuit boards, electrical components, mechanical pans, 
containers, bottles, automotive parts, paper goods, etc. 

15 Figure 3 depicts a machine vision system 80 according to the invention for " 

determining the reference point and orientation of an object 82 having coupled thereto a 
calibration target 84 according to the invention and, particularly, a four-region target of 
the type shown in Figure IB. The system 80 includes an image capture device 86 that 
generates an image of a scene including object 82. Although the device may be 

20 responsive to the visual spectrum, e.g., a conventional video camera or scanner, it may 
also be responsive to emissions (or reflections) in other spectra, e.g., infrared, gamma-ray, 
etc. Digital image data (or pixels) generated by the capturing device 86 represent, in the 
conventional manner, the image intensity (e.g., contrast, color, brightness) of each point in 
the field of view of the capturing device. 

25 

That digital image data is transmitted firom capturing device 86 via a 
conrniunicaiions path 88 to an image analysis system 90. This can be a conventional 
digital data processor, or a vision processing system of the type commercially available 
fi-om the assignee hereof; Cognex Corporation, as programmed in accord with the 
30 teachings hereof to determine the reference point and orientation of a target image. The 
image analysis system 90 may have one or more central processing units 92, main memory 
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94, input-output system 96, and disk drive (or other mass storage device) 9«, all ot the 
conventional type. 

The system 90 and, more particularly, central processing unit 92, is configured by 
5 programming instructions according to teachings hereof for operation as illustrated in 
Figure 4 and described below. Those skilled in the art will appreciate that, in addition to 
implementation on a programmable digital data processor, the methods and apparatus 
taught herein can be implemented in special purpose hardware. 

10 Referring to Figure 4 there is shown a machine methodology according to the 

invention for interpreting an image of a target 84 to determine its reference point and 
orientation. The discussion that follows is particulariy directed to identifying a four- 
region target of the type shown in Figure IB. Those skilled in the art will appreciate that 
these teachings can be readily applied to finding targets according to the invention, as well 

15 as to other targets having detectable linear edges that are oriented toward a reference 
location or reference point on the target, e.g., a prior art cross-shaped target. For 
convenience, in the discussion that follows, such linear edges are referred to as "adjoining 
edges," regardless of whether they are firom calibration targets according to the invention 
or from prior art calibration targets. 

20 

In step 100, an image of the target 84 (or of the target 84 and object 82) is 
generated, e.g., using image capture device 86, and input for machine vision analysis as 
discussed below. The image can be generated real time, retrieved from a storage device 
(such as storage device 98), or received fi-om any other source. 

25 

In steps 102 - 108, the method estimates the orientation of the target in the image 
using any of many alternative strategies. For example, as shown in step 102, the method 
determines the orientation by applying a conventional Hough line vision tool that finds the 
angle of edges discemable in the image. In instances where the target occupies the entire 
30 image, those lines will necessarily correspond to the adjoining edges. Where, on the other 
hand, the target occupies only a portion of the image, extraneous edges (e.g., from other 
targets) may be evident in the output of that tool. Although those extraneous edges can 

10 
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generally be ignored, in instances where they skew the results, the image can be 
windowed so that the Hough vision tool is only applied to that portion that contains the 
target. Once the angles of the lines has been determined by the Hough line tool, the 
orientation of the image is determined from the predominant ones of those angles. 
Alternatively, the angle of the image can be determined by taking a histogram of the 
angles. 



The Hough vision tool used in step 102 may be of the conventional type known 
and commercially available for finding the angle of lines in image. A preferred such tool 
10 is the Cognex Line Finder, commercially available from the Assignee hereof, Cognex 
Corporation. 



An alternative to using a Hough vision tool is shown in step 106. There, the 
illustrated method determines the orientation of the target by applying a Sobel edge tool to 
15 the image to find the adjoining edges. Particularly, that tool generates a Sobel angle 
image that reveals the direction of edges in the image. As above, where the target 
occupies the entire image, the adjoining edges will be the only ones discerned by the 
Sobel edge tool image. Where, on the other hand, the target occupies only a portion of the 
image, any extraneous edges can be ignored or windowed out. 

20 

The Sobel edge tool may be of the conventional type known and commercially 
available for finding lines in image. A preferred such tool is the Cognex Edge Detection 
tool, commercially available from the Assignee hereof, Cognex Corporation. 

25 Once the Sobel angle image is generated, in step 106, the orientation of the target 

in the image is determined by generating a histogram of the edge angle information; see, 
step 108. From that histogram, the target orientation can be determined by taking a one- 
dimensional correlation of that histogram with respect to a template histogram of a target 
oriented at 0°. Where a Sobel magnitude image is generated, in addition to the Sobel 

30 angle image, such a histogram can be generated by counting the number of edges greater 
then a threshold length at each orientation. 
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As a Still further alternative to applying a Hough vision tool or Sobel edge tool, the 
method contemplates obtaining the angle of orientation of the target from the user (or 
operator). To this end, the user may enter angle orientation information via a keyboard or 
other input device coupled with digital data processor 90. 

5 

In steps 110 - 118, the method determines the location, i.e., coordinates, of the 
target reference point in the image. Particularly, in step 110, the method can apply a 
Hough vision tool, as described above, to find the angle of lines discemable in the image. 
A conventional Hough vision tool determines, in addition to the angle of lines in an image, 
10 the distance of each line, e.g., from a central pixel. As above, where the target occupies 
the entire image, those lines will be the only ones discemable by the Sobel edge tool 
image. Where, on the other hand, the target occupies only a portion of the image, any 
extraneous edges can be ignored or windowed out. 

15 As above, the Hough vision tool used in step 104 may be of the conventional type 

known and commercially available for finding the angle and position of lines in image. 
Once again, a preferred such tool is the Cognex Line Finder, conmierciaily available from 
the Assignee hereof, Cognex Corporation. Those skilled in the art will, of course, 
appreciate that steps 102 and 1 10 can be combined, such that a single application of the 

20 Hough vision tool provides sufficient information from which to determine both the 
orientation of the target in the image and its reference point. 

As an alternative to using a Hough vision tool, the method can apply a projection 
vision tool to the image in order to find the position of the lines discemable in the image; 

25 see, step 1 12. The projection tool, which maps the two-dimensional image of the target 
into a one-dimensional image, is applied along the axes defined by the edges in the image. 
As those skilled in the art will appreciate, the location of the edges can be discerned from 
by finding the peaks in the first derivatives of each of those projections. As above, where 
the target occupies the entire image, those lines will be the only lines discemable by the 

30 Sobel edge tool image. Where, on the other hand, the target occupies only a portion of the 
image, any extraneous edges can be ignored or windowed out. 
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The projection vision tool used in step 1 12 may be of the conventional type known 
and commercially available for mapping a two-dimensional image of the target into a one- 
dimensional image. A preferred such tool is that provided with the Cognex Caliper tool 
commercially available from the Assignee hereof, Cognex Corporation. 

In step 1 14, the method uses the information generated in steps 1 10 and 1 12 to 
compute the location of the reference point, particularly, as the intersection of the lines 
found in those steps 1 10 and 1 12. 



10 As an alternative to using the Hough vision tool and the projection tool, the 

method can apply determine the location of the reference point by performing a binary or 
grey scale correlation on the image; see step 116. To this end, the method uses, as a 
template, a pattern matching the expected arrangement of the sought-after edges, to wit, a 
cross-shaped pattern in the case of a target of the type shown in Figure IB. The use of 

15 correlation vision tools for this purpose is well known in the art. The template for such an 
operation is preferably generated artificially, although it can be generated fi-om prior 
images of similar targets. 

As still another alternate to the Hough vision tool and the projection tool, the 
20 method can apply a grey-scale image registration using the sum of absolute differences 
metric between the image and a template; see step 118. To this end, the method uses, as a 
template, a pattern matching the expected arrangement of the sought-after edges, to wit, a 
cross-shaped pattern in the case of a target of the type shown in Figure IB. The template 
for such an operation is preferably generated artificially, although it can be generated from 
25 prior images of similar targets. A preferred grey-scale image registration tool is disclosed 
in United States Patent No. 5,548,326, the teachings of which are incorporated herein by 
reference. 



Although steps 110-118 can be used to determine the approximate location of the 
30 reference point of the target in the image, the method utilizes optional steps 120 and 122 
to refine that estimate. These two steps are invoked one or more times (if at all) in order 
make that refinement. 
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In step 120, the method applies a conventional caliper vision tool to tind points in 
the image that define the adjoining edges of the regions. On the first invocation of step 
120, the method applies calipers along fifty points along each edge (though those skilled 
in the art will appreciate that other numbers of points can be used), beginning with points 

5 closest to the estimate of the reference point, as discerned in steps 1 10 - 118. The calipers 
are preferably applied a small distance away fi'om the actual estimate of the reference 
point to avoid skewing the analysis due to possible misprinting of the target at that point, a 
missing pattern at that point (e.g.. Figure IC), or a too-high spatial fi-equency at that point 
(e.g., Figure IB). In step 120, the method then fits a line to the points found along each 

10 edge by the caliper tool, preferably, using a conventional least squares technique. 

In step 122, the method computes a refined location of the reference point as the 
intersection of the lines identified in step 120, Although conventional geometric 
calculations can be performed for tliis purpose, preferably, the reference point is computed 

15 using conventional least squares techniques. Preferably, when examining the image of a 
symmetric calibration target of the type shown in Figure IB, the method utilizes the same 
number of points on either side of (and closest to) the reference point for purposes of 
fitting each line. This minimizes the bias otherwise introduced by a conventional edge 
detection technique in finding edges that are defined only by dark-to-light (or light-to- 

20 dark) transitions. 

Calibration targets of the type shown in Figure IB are advantageously processed 
by a method according to the invention insofar as they fiirther minimize bias otherwise 
introduced by a conventional edge detection techniques. In this regard, it will be 

25 appreciated that such bias is reduced by the fact that "opposing" adjoining edges (i.e., 
edges that oppose one another across the reference point) define straight linear segments 
that change polarity across the reference point. That is, those segments are defined by 
regions that transition — preferably, equally in magnitude - from light-to-dark one one 
side of the reference point, and firom dark-to-light on the other side. This is true for all 

30 "symmetric" calibration targets according to the invention, i.e., targets in which opposing 
edges define straight linear segments that are opposite polarity on either side of the 
reference point. 
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On the second and subsequent invocations of step 120, the method speciticaliy 
applies the caliper tool at points along the lines found in the previous iteration. 
Preferably, these points are at every pixel in the image that lies along the line, though, 
those skilled in the art will appreciate that few points can be used. The calipers can be 

5 applied orthogonal to the lines, though, in the preferred embodiment, they are applied 
along the grid defined by the image pixels. The caliper range decreases with each 
subsequent invocation of step 120. The method continues applying the calipers, beginning 
at the estimated center point until one of the four following situations occurs: no edge is 
found by the caliper applied at the sample point; more than one edge is found by the 

10 caliper and the highest scoring edge is less than twice the score of the second highest 
scoring edge (this 2X comes from the CONFUSION^THRESHOLD); the distance 
between a computed edge point and the nominal line (computed from the previous 
invocation of step 120) is larger than a threshold (which threshold decreases with each 
subsequent invocation of step 120); or, the caliper extends outside of the image. As 

15 above, in step 120, the method then fits a line to the points found along each edge by the " 
caliper tool, preferably, using a conventional least squares technique. 

In the second and subsequent invocations of step 122, the method computes a 
refined location of the reference point as the intersection of the lines identified in step 120. 
20 As above, although conventional geometric calculations can be performed for this 
purpose, preferably, the reference point is computed using conventional least squares 
techniques. 



Referring to Figure 5, there is also shown a machine methodology according to the 
25 invention for interpreting an image of a target 84 to determine its reference point and 
orientation. Particularly, in step 150, the method calls for generating an image of a target 
84 and, particularly, of a target according to the invention having two or more regions, 
each region being defined by at least two linear edges that are directed toward a reference 
point, and having at least one of the regions having a diflFerent imageable characteristic 
30 from an adjacent region. Step 152 can be effected in the manner described in connection 
with step 100 of Figure 4, or equivalents thereof 
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In step 152, the method analyzes the image to generate an estimate of an 
orientation of the target in the image. Step 152 can be effected in the manner described in 
connection with steps 102 - 108 of Figure 4, or equivalents thereof 

5 In step 154, the method analyzes the image to generate estimate of a location of the 

target's reference point. Step 154 can be effected in the manner described in connection 
with steps 1 10 - 118 of Figure 4, or equivalents thereof 

In step 156, the method analyzes the image to refine its estimates of the location of 
10 the reference point in the image and the orientation of the target in the image. Step 1 56 
can be effected in the manner described in connection with steps 120- 122 of Figure 4, or 
equivalents thereof 

Calibration target and methods for analysis according to the invention are 
15 advantageous over prior art targets and methods insofar as they are magnification 

invariant. By analyzing the adjoining edges of targets, methods according to the invention 
insure reliance on features (to wit, regions) that retain the same imageable appearance 
regardless of magnification. This is in contrast to prior art targets and methods, which rely 
on individual lines (or dots) to define calibrating features. As noted above, the imaging 
20 appearances of such lines and dots change with varying degrees of magnification. Even 
the prior art methods that analyze checkerboard targets rely on analysis of comers, which 
are not magnification invariant. 



The magnification invariance of targets and methods according to the present 
25 invention is illustrated in Figures 6A - 6C. Referring to Figure 6 A, there is shown an 
imaging setup wherein camera 200 images a target 202 (on object 204) fi-om a height x. 
An image generated by camera 200 is displayed on monitor 206 of workstation 208. 
Referring to Figure 6B, there is shown an identical imaging setup, except insofar as a 
camera (of identical magnification) images a target (of identical size) from a greater 
30 height, x'. Likewise, Figure 6C shows an identical imaging setup, except insofar as a 
camera (again, of identical magnification) images a target (again, of identical size) fi-om a 
still greater height, x". Comparing monitor images depicted in Figures 6A - 6C, it is seen 
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that the images of the targets, generated by the cameras and displayed on the monitors, are 
identical in appearance regardless of the relative heights (x, x', and x") of the camera. This 
magnfication invariance results from the fact that the target has the same appearance 
regardless of height (or equivalentiy, magnification), within specified limits thereof 
5 Those skilled in the art will appreciate that those limits are greater than the analagous 
limits for prior art targets (e.g., cross hairs, parallel stripes, etc.). 

Still another advantage of calibration targets and methods according to the 
mvention is that they permits angular orientation to be determined throughout a fiill 360° 
10 range. With reference to Figure ID, for example, the relative positions of the regions can 
be used to determine the overall orientation of the target, i.e., whether it is rotated 0°, 90<*, 
1 80°, or 270°. This information can be combined with a determination of relative 
onentation made by analysis of the adjoining edges as discussed above to determine the 
precise position of the target. 
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APPENDIX I 



Patent Application for 
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MACHINE VISION CALIBRATION TARGETS AND METHODS 
OF DETERMINING THEIR LOCATION AND ORIENTATION 
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Vision Tool Description 



THE FOLLOWING APPENDIX IS NOT BELIEVED TO BE NECESSARY FOR 
25 ENABLEMENT OR BEST MODE DISCLOSURE OF THE INVENTION DISCLOSED AND 

CLAIMED IN THE ACCOMPANYING APPLICATION. 
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Edge Detection Tool 

This chapter describes me Edge Detection tool, which includes edge detection ana peak 
detection. 

Edge detection takes an input tmage and produces two output images: an innage ot the 
edge magnitude of eacn Input pixel and an image of the edge angle of each inpu pixel 
When combined with other Cognex software toots, the information produced by edge 
detection can be used to locate oDiects wiihrn an image. Edge pixel information can be 
used to detect the rotation of an ob)ect or detects such as scratches, cracks, or particles. 
You can also gauge now snarpty an tmage is focuseo with edge pixel information. 

PoQk dmtQction is uCGtuI in any appiicanon m which knowtodgo of trio local maximum valuoe 
in a two-dimenstonat soace is useful. Peak detection takes an input image and produces 
an output image containing only those pixels in the input image with higher values than 
neighboring pixels. A typical input image to peak detection is an edge magnitude image: 
the cutout image contains only the highest magnitude edge pixels. The ecge oetection 
function can oooonaliy use peak detection to postprocess the edge oetection results so 
that only the strongest eages remain. 

This chapter has nirte sections as follows: 

An Ovennew of Edge Detection describes the goals of edge detection, defines an edge 
pixel, and describes now the Edge Detection tool finds edge pixels. This section atso 
axQtatns the two oropenies of a pixel that are calculated by the Edge Detection tool, eage 
magnitude and angle, and ends with a sample application. 

Using Edge Oerecr/oo describes, in general terms, the interface to the Edge Detection toot. 
THIS section contains a aiscussion of the compression tables mat affect the amount of 
memory used by the eage detector. 

Edge Detection Enumerations and Data Structures oescrioea the data structures and 
enumerations that the edge detection functions use. Types and data structures that 
support peak detection are discussed in Peak Detection Enumeration Types and Data 
Structures below. 
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4 Edge Detection Tool 



Edge Dataction Functions deacribes the functions mat impl©ment eage oatoction. 

Maximizing the Performance of VC2 Vision Coprocessor \\st^ tne edge detection parameter 
aerunga thai you should avoia i! you want to maxtmiie the oerformance ot tne VC2 vision 
coprocessor dunng edge detection. 

Overview of Peak Detection defines oeaK pixel and gives examples of some applications 
of peak dotection. 

Using Peak Detection describes the intertace to peak detection. 

Peak Detection Enumeration Types and Data Structures describes the peak detection 
enumerations ana aaia structures. 

Peak Detection Fu/icf/o/Js d««crib« \he poak dotoction functionc. 
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Edge Detection Tool 4 



Some Useful Definitions 



•dg« plx«l 
comprM«k>n Ubiet 



SotMl operators 

horizontal wise 
component 

v«rtlc«t edgo componont 
edge magnttude 
edge engle 



peek pixel 

peek detection 

8-point nelghboHiood 

4-polm neighborhood 
2-po»m nelghboTtiood 



Pixel that nas a different value from one or more of its 
etgnt neignoorir^g pixels. 

Operator that finds eago pixolc in an image. 

Two tables thai are created during edge detection 
inttializaoon: the edge compression table and the 
magmrude compression table. These tables ere used by 
the Edge Detection too) to mao data into a smaller range 
or values. 

Two 3x3 operators mat the Edge Detection tool uses to 
\acztB edge pixals in an image. 

Value reojrned by the horizontal Sobel operator when it is 
applied to a pixel in an image. 

Value returned by the vertical Sobel operator when it is 
applied to a pixel in an image. 

value tnat increases as the ditterence tn grey levels 
between neighboring pixels increases. 

Orientation of an edge, with respect to the x-axis of the 
image, at an edge pixel. The Edge Detection tool defines 
me angle of a pixel as arctan(v/h). wnere v^is the vertical 
edge component and h is the nonzontat edge component 

Of the pixel. 

Pixel with a value greater than or equal to some or all of 
Its neiohbortng pixels' values. 

Finding some or all peak pixels in an image. 

Pixels mat are horizontally, vertically, and diagonally 
ad)acent to a pixel. 

Pixels that are horiromally and vertically adjacent to a 
pixel. 

Neignbas of a pixel along a single axis. PoaK detection 
supports a horizontal and a venical 2-point neighborhood 
operator. 
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4 Edge Detection Tool 



•yvnmetnc ptMk 
asymmetric p«ak 

platMu 



Peak wnose pixel value is greater than or equaJ to the 
values or each of its rwighbors. 

Peak wnose pixel value is greater than rr»e values of its left 
and low«r noightoorc and groaior than or oquol to tho 
values of lis right and upper neighPors. 

Pixel that satisfies the peak definition (symmetric or 
asymmeuic) along all axes in its neighborhood. 

Pixel that sansfies the peak definition (symmetric or 
aaymmetrio) along at toast one axis m ita naighborhood. 

In peak detection, a region of conoguous pixels of the 
same value. 
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Edge Detection Tool 4 

An Overview of Edge Detection 

The Edge Detection lool finds eage pixels in an image. This overview of edge deieciion 
oontainc tho following daccnpiionc: 

• Definnion of the problem of edge detection 

• Overview of trie tasks tr^ai me Edge Detection tool pertorms 

• Definition of edge pixel 

• Deschonon of the SoOe) operator, which is used by tne Edoe Detection tool to locate 
edge pixels 

Oefiniiiona of tho angle and mQgmtuae of on 0090 pixel, and deacriptione of the two 
images produced by the Edge Detection tool: the magnitude image and the angle 

« Description of edge detection preproces&ing 

• Description of edge detection postprocessing 

• Description of a sample application that uses both the magnitude and me angle 
images to create an angle "signature' of a shape 

Edge Detection 

Tr^e Edge Detection tool locates edges and determines each edge's angle and magnitude. 

An edge occurs wherever adjacent areas of an image have different grey values. Using this 
definition the triangle in Figure 66 has a single edge. 




Figure 66. A figure with an edge 
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4 Edge Dttection Tool 



The definition of edge can be further refined as the directed border between grey areas of 
an image, where direction is defined as a vector normai to the edge. Using this definition, 
the trienglG in Figure £7 hfis three airectional edges, repracanted by the three vcctorc. Tno 
angle of the edge is the counterclockwise angle of the vector normal to the eage. with 
resoeci to the horizontal axis of me image. See the section Angie Imaoe on oage 1 81 for a 
discussion of edge angle. 




^tguro e/. Vectoi^ tnoicating the eage engtes of a thangte 

Along with an angle, an edge has a mBgnttuda. which reflects the amount of the difference 
between grey levels on either side of the edge. As ihe dttterence in grey levels increases, 
(he maonttude increases. See the section The Magnitude tmage on oage 176 for a. 
discussion of edge magnitude. 

The triangle in Figure 68 has the same three edges as the triangle in Figure 67: however, 
these edges are of lower magnitude. This lower magnitude is Illustrated by the shortenea 
lengm of me three direction vectors. 




Figure 68. A triangle with iow-magniiuda edges 
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Edge Detection Tool 4 



Edges are actually iocated pixel by pixel. Figure 69 contains a magnitied view of a portion 
of an edge m an Image. Each cell in the figure reoresents ihe grey level of a single pixel. 
Notice l^at. on this "micfoscooiC level, there are many edges between pixels »n which the 
grey levels on either side ot the edge differ by a tew percentage points. 
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figure 60. Croy ievQtc tn a magnifiod poeiton of an odgo 

The Edge Detection tool locates edges in an image by identifying each pixel that has a 
different value from one of its neighboring pixels. It calculates the angle ana magnitude of 
eacn edge pixel. It also provides a means of classifying edges, so that lowz-magnitude 
edges are not reported. 
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4 Edge Detection Tool 



Edge Pixels 



A pixol iG an octga ptxoi\\ it haa a ditforam vaiua from at leaet one of ite eight neighboring 
Dixeis ana is noi on the border of an image. In Rgure 70, the four snaaed pixels are edge 
oixals. 
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Figure 70, The shaded pixels are edge pixels 

Figure 7i contains a g'id representing grey levels in an in^age with the highest magnitude 
edge pixels shaded. Notice that border pixels (pixels along the eoge of the image) are not 
edge pixels. 
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figure 71. Highest magnttudo edge pixels in en imege 
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Edge Detection Tool 4 



The Edge Detection Operators 

The Edge Detecuof^ loot finds «cg6 pixolc using tno So^3fi/3x3 naighbornood opdratorc. 
During edge detection, the SoDei operators are applied to each pixel In an input image. 
This ooeraton produces two values for each pixel. One value raDrasenis the vortical Gdge 
component tor the pixel; the other value represents the horizontal e0ga component tot the 
pixel. These two values are then used to comoute tne eage magnitude and edge angle of 
the pixel. 

Sobel edge detection uses two 3x3 neighhornood operators to locate edge pixels In an 
tnnage. The horizontal Sobet operator detects the horizontal (x) edge component for a pixel. 
Tne vertical Sooei operator aatecis tne venicai (y) eoge comooneni for a oixei. Figure 72 
shows the Sobet operators. 
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Figure 72. Sabef operators 

The Sobel operator works only on pixels with eight neighboring pixels. When the Sobel 
operator is applied to a pixel on the border of an image, me result is defined to be 0 
because there are not enougr> neighboring pixels to calculato an odgo value. 

The horizontal and vertical Sobel operators are 3x3 linear operators. These operators 
compute a value for a pixel (x.y) in the following way: 

• The grey level of the pixel (x.y) is muiiiplied by the value in the center of the 3x3 linear 
operator. 

• The grey level of the neighboring pixel (x-ly-l) is multiplied by the first value in the 
top row of the 3x3 linear operator. 

• The grey level of the neighboring pixel (x,y-t) is multiplied by the second value in the 
top row of the 3x3 linear operator. 

• This continuos untd ths produot of ©aoh pair of veluee ie calculated. 

• These products are then surrwred to produce the output. For the Sobel operator the 
value IS interpretea as horizontal or vertical eage component or tne pixel 
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4 Edge Detactlon Tool 



If tiie result of applying at )eas{ one of me Sobet operators to a pixel is nonzero, that pixel 
IS an edge pixel. 
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Figure 73. Applying the Sobel operator 

in Figure 73, the horizontal Sobel operator is applied to the shaoed pixei in the image. The 
resultant edge pixel value is 6. whicn is calculated as follows: 
-l-UO-Ul-2-i--2 1r02T23^.12*03*l-3 = 6 

A/ore; The range of pixel values in Figure 73 and in most examples in this docifnent is much 
smaller than the typical range of pixel values in an image. This is done to simplify the 
examples. 

Ftgure 74 sha>A/s fill the vertical and horizontal oogo componont valuac for a email imagtt. 
The upper grid is the image: the values represent the grey levels of each pixel. The gria to 
the lower left of the image contains horizontal edge comoonem values for the image. Each 
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cell in the grid coniains the result of appfying the Sobel horizontal opefator to the 
corresponding input image pixel. The grid to the lower right contains the vertical cage 
comoonenis tor the inr^age. tt is conr\Duted using the Sobel vemcal ooerator. 
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Horizontal edge connponem values 



Vertical edge component values 



Figure 74. Horizontat ana vertical edge componenr vaiues computed 
with the Sobei operators 

The horizontal and vertical edge component values computed with the Sobei operator are 
not returned by the edge detection functions. They are used to compute the edge 
magniruae ana edge angie or eacn pixel in tne image. 
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The Output images 

£dgo magnitudafi aro ctorafl \n an outoui imago that ehowa m« <>dga magnitudo o* oach 
corresponding pixel in the input image. Edge angles are stored in another output image 
that Shows the edge angle of each pixel. You can chooso wnicn outout imagec to create; 
magnitude, angle, or both. 
These two Images are described in ihis section. 

Ihm Magnttuds Imag* 

The eage magnitude for a given pixel is a function of the horizontal and vemcal edge 
components of the pixel. If x is the hortzomal edge component value and y is the vertical 
eage component value tor some pixel, then the eage magniiuoe. M. for tnat pixel is c3aTir>ea 



as follows: 




This formula suggests a geometric Inter preiation of the data: it xis the horizontal edge 
component and y is the vertical edge component, me magnitude is a vector from tne origin 
of a two-dimensional coordinate system to the point (x.y). Figure 75 shows the geometric 
relationehip between the edge magnttud© and the two edge oompononia. 




X Horizontal edge 
component 



Edge magnitude 



Figure 76. Geometric repreeentetion of edge megrtitude 

Note- For imolementation-soecific reasons, the Edge Detection toot uaes the formula 
described above lo compute edge magnitude and then scales the magnitude upward 
by approximately 16%. 
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You can cnoose the numDer of bits mat can be used to express magnitude. Because 
magnitude values, comouted witn vertical and horizontal edge component values, can 
o^eed itils number o1 bits, maonitude values are comoressed to an tnteoer value tnai can 
De expressed in the number oi bits you have specified. You can control now magnitude 
values are comoressed: you can choose a method suooiied by the Edoe Detection tool, 
such as a logarithmic or linear map. or you can supply a magnitude compression map of 
your osvn. For descriptions of the maps supplied by the £dga Detection tool and for 
information on using your own map. see the section Two Compression Tables on page 192. 

The size and depth of the ma^itude image ere the same as that of the input image. Since 
aU border pixels have horizontal and vertical edge pixei values of 0. the border of the 
magrilude image contains all zeros. You can display an edge magr«tude image: higher 
magnitude edges are brighter. 

Optionatiy, edge detection can return the numoer of edge pixels in tne input tmage and the 
sum of all edge magnitude values. By dividing the sum of all edge magnitude values by the 
size of the image, you can compute tne average edge magnitude of an image. If you have 
two Images of the same scene, you can determine which image is in sharper focus by 
comparing average edge magnitudes of the two images: the image with the higher average 
magnitude is in sharper focus. 
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Figure 76 contains an image, the image's horizontal and vertical edge components 
comouted with Sobei operators, and the output image containing the edge magnitude 
values for each pixel. The edge oixels with the highest magnitude, representing the actual 
edge, are shaded. 
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Figure 76. Edge magniiuaa values computed wirh me Sobei operators 



The edge detection funottona produce an image oontaining tno angle of each eogo pixel. 
If /is the honzontfll edge component value end /is the vertical edge component value, the 
edge angle te the oountorclockw/ifte angio betwwdon the honiontai axis and me magnitude 
vector. 

Figure 77 snows tne geometric reiationsnip oetween the edge angle, me eage magnitude, 
and the two edge components. In this figure. M is tne magnitude vector and 6 is the edge 
angle. 



Vertloal edge 
component 




X Horizontal edge 
component 



Figure 77. Geometnc interpretation of edge angle 

Edge angles are stored In the angle image m Dinar/ ^orm. They are represented as 
fixed-point numoers with tne decimal point to the leli o( tne most significant bit and are 
interpreted as fractions of 360**, You can choose the maximum numoer or dies avaiiaoie to 
express an angle: this value must be less than or equal to 8. If n is the maximum numDer 
of bits used to express angle, ana x is the binary representation of an angle, you can 
convert / to degree notation oy using tne following formula: 



angle(x) 



360x 
2« 



Note that with a 6-bit representation, angles can be expressed to the nearest 5,626" 
a 7-blt representation, an angle can oe expressed to the nearest 2.8": with an 8-bn 
representation, an angle can De expressed to me nearest 1.4^ 



wilh 
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The odge angle is computod using the arc tangent function if x is the horizontal edge 
comoonent and y is the vertical edge component to*- a pixel, the eage angle for that pixel 
la cQtoulatod oc shown in Tablo 3. 



H 


M 


Edo« Angla 


x> 0 


y > 0 


arctan( y/x ) 


X > 0 


y < 0 


360-arctan( ty/xl ) 


X > 0 


y=0 


0 


X < 0 


y > 0 


ieo-arctan( ly/xi ) 


X < 0 


y < 0 


ISO arctan< y/x ) 


X < 0 


y = 0 


180 


x-0 


y > 0 


90 


x = 0 


y < 0 


270 


x==0 


y a 0 


0 



Table 3, Formutse for computing edgB angles 
The computation is designed so that an edge angle e is in me roiiowmg range: 
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Figure 78 contains six rows. Each row contains (from left to ngnt) an image with the central 
pixel shacied. the vertical and horizontal edge component value of the shaded pixel, the 
(mage witri a vector auponmposod showing th« »dgA anglo of the contra! pixol. and tho 
lormula used to compute trte edge angle. Notice that the equatior^s defining cne eoge angle 
vaiuA are designed so that tho vector always ooints to the OriQhter side of tne edge. 



Vertical edge = *4 
Horaonial edge = 0 



E1I3 

nokiDo 
Danno 



Angle = arc tan(-d/0) = 270" 



Vertical edge s 0 
Horizontal edge « 4 



□□□DO 

QiD .nn 



T] Angle = arc tan(0i/4) ^ 0' 



Vertical edge - -3 
Horizontal edge « -3 
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OQQEIB 
DD 

DLDQEl 
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Angle = arc ian(-.3/3) = 225' 



Veaical edge « 1 
Horizontal edge = i 



□noa 
□□ono 

□□□□□ 



Angle = arc tan(1/1) = 45° 



Vertical edge = 4 
Horizontal edge = 0 



nnDOD 
DDnaa 

□□□□□ 



Angle = arc tan(4/0) = 90' 



Vertical edge = 3 
Horizontal edge = -3 



nn . rara 

DBGIGIia 



Angle = arc ian(3/-3) ^ 135° 



Figure 78. Comouitng rn© angle ot vanous edgas 
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By aetung a flag in tna edge detection parameiers. you can spectty thai you want angles 
computed in the range 0* to 1 80'* insteaa of the default range. 0" to 360". When you selea 
tno small«r range, an edge angle nthat is greater than 1 80*' maps to edge angle (n— 180**) 
For instance, 300** maps to 120**. and 270** maps to 90*, 

Use The reduced angle range when you want tne same edge angle results regardless of 
the polarity of the image. As shown in Figure 79, when the range of angles is restricted to 
tne range 0** to ISO**, the same edge angles are computed tor a black object on a white 
background as tor a white obiect on a black background. When the range of angles is 
to 360" (also Figure 79) this is not the case: reversing the polaniy of the image reverses the 
edge pixel angle. 




Figure 79. Eftect oi reversea poiarny on edge angles lor the two ranges 

The binary representation of an angle in the range 0' to 1 80** is similar to the representation 
of an angle in the range C* to 360*. if n is tne maximum number of bits used to express 
angle and xls the binary representation ot an angle in the range 0** to 180**. you can convert 
xxo dogroo notation by applying thfi following formula: 
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Edge Detection Preprocessing 

If you supply a pixel map. the edge detection tunctian usee it to proproGo» your inpui 
image before li calculates me horizomat and varacal edge ccmDonents. For a aescnption 
of Dixat mapping. e©o Chapter 3, f^ixQi Mapping. 

Edge Detection Postprocessing 

Aimough edge angles are defined for all pixels in an image, l^ey are signilicant only at the 
pixels witn the greatest edge magnitudes. Thafa are many edges petween pixels wnere the 
grey levels on either aide of the edge differ by only a faw percent. Theae inaignifioant edge 
pixels, due to frame grabber noise and random fluctuations in pixel values, can clutter your 
edge magnitude imago and make it difficult to evaluate tne data in that imago. Aioo odge 
magnitude values usually increase gradually in a "smear* as they approach an edge and 
then docroaM ac the edg* ic croccod. You might want to charpon tho magnitude imaga by 
filtering out some edge pixels that are near, but not at, an edge. 

The Edge Detection tool supplies three postprocessing methods. Two of these methods, 
setup and arn-r/me thresholding, set the lowest edge magnitude pixels to 0. The third 
method, peak detecrion, takes an eage magnitude image ana processes it so mat. tor any 
smalt pixel neighbahood. only the highest edge magnitude pixels remain and the other 
pixels are set to o. See tne section pbbk OBtecuon on an Eage Magnituae Image on 
page 229 (or a complete discussion of this method. 

After postprocessing, edge angles corresponding to zero edge magnitude pixels are also 
set to 0 unless you have specified run-time thresholding but not peak detection. In this 
case, edge angles corresponding to zero edge magnitude pixels can be nonzero, but 
should be considered undefined, 

Satup Thresholding 

You can supply a setup time rejection percentage. This Is a value n that represents a 
percentage of the highest possiDie edge magnitude value. With this method, any 
magnitude values in the first n percent of at! possible values are forced to 0. For example, 
if you provide 6 bits to express magnitude values and specify that the lowest 5 percent of 
magnitude values are forced to 0. then any magnitude Delow 3 is forced to 0 since 5 
percent of 63 (truncated) is 3. 

Run-Timo Thresholding 

You can supply a runMtme rejection percentage. This is a value n that represents a 
percentage of the highest ec/ua/ edge magnitude value in a magnitude image. With this 
method magnitude values in the first n percent of the magnitude values in a specllic image 
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ara forced to 0. For examole. if the highest magnitude value in a particular image ts Si . and 
if you specify that the lowest 5 percent of the run-time magnitude values are forced to 0. 
then any maonitude below 2 is forced to 0. since 5 percent of 51 (truncated) is 2. 

Using a setup rejection percentage is faster than using a run-time rejection percentage, 
since the values that are lorcad to 0 are computed once, before any magmiuoe images are 
created. However, the run-time method is more flexible and robust. If you are producing 
magmtuoe images or the same scene out under varying lighting conditions, you snouia use 
a run-time rejection percentage since the proportion of magnitude values forced to 0 
remains stable as the range of magnitude values changes (due to varying illumination). 

PostprocMstng with PMk Ootoction on a uagnnuda knmgm 

Pettk dstoction is a method of filtering en image 90 that only peak pixoi3 remmn and other 
pixels are set to 0. Peek pixels have a value greater than or equal to some or all of their 
noighfooring piKeie' valuea. 

You can set a flag in the edge detection parameters structure that instructs the edge 
detection function to run peak detection on the magnitude Image during postprocessing. 
All nonpeak pixels are filtered out of the magnitude image and the edge angle image before 
they are returned. When you specify this, you must supply peak detection parameters in 
the edge detection parameter structure. 

The edge deteaion function does peaK detection postprocessing after it has done setup 
and run*time throaholding. There wtll be a 2-pixel -wide bordar of zeroe in a poak-doteotod 
magnitude image. 

The section Peak Detection on an Edge Magnitude Image on page 229 discuEses. in detail, 
the benefits of using peak detection on an edge magnitude image. This section also 
describes the proper way to parametanze peak detection so that it fitters edge magnitude 
pixels correctly. However, you do not need a detailed knowledge of peak detection to use 
it with edge detection. Suggested peak detection parameters are supplied tn the section 
Data Structure cip_edge_params on page 199. 

Figure 80 contains an image, its edge magnitude image, and the image that results from 
peak detection. The edge magnitude image contains a smeared edge, which is sharpened 
by peak detection. 



38 



SUBSTITUTE SHEET (RULE 26) 

BNSOOCID: <WO__98iei 17A2_L> 



wo 98/18117 



PCT/US97/18268 



Edge Detection Tool 4 



0 


0 


0 


0 


0 


0 


0 


1 


0 


0 


0 


0 


0 


0 


1 


3 


0 


0 


0 


0 


0 




3 




0 


0 


0 


1 


3 


4 


4 




0 


0 


1 


4 


4 


4 


4 




0 


1 


3 


4 


4 


4 


4 




1 


3 


4 


4 


4 


4 


4 




4 


4 


4 


4 


4 


4 


4 





Grey levels 



Peak edge pixeb 



0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


19 


0 


0 


0 


0 


0 


18 


19 


0 


0 


0 


0 


0 


18 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 



0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


2 


8 


16 


0 


0 


0 


2 


7 


14 


19 


16 


0 


0 


2 


10 


16 


10 


14 


7 


0 


0 


8 


16 


16 


7 


2 


0 


0 


0 


16 


16 


7 


0 


0 


0 


0 


0 


16 


8 


2 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 



Edge magnitude pixels 



Figure 80. Peak detecting an edge magnitude image to determine actuai 
edge 



39 



SUBSTmJTE SHEET (RULE 26) 



BNSDCX:iD: <WO 981Bn7A2J_> 



wo 98/18117 



PCT/US97/18268 



4 Edge Detection Tool 



Sample Application: Angle Histogram 



You con uee the magnitude ar>d angle imegoo of an obioot lo oroate an angio histogram of 
mat object. This is a tabte inat assigns a value n to each edge angle 6 if there are nedge 
aixalc (abovo a givon magnituda throshold) umoaa edge angle is a. 

The angle hisiogran^ supplies a useful "signature' o< an object's shape. You can use this 
signature to gauge similarity anr\ong objeas. to detect rotations of an obioct. and to detect 
flaws in an object. 

Once you have created edge magnitude ard edge angie images for an object, you can 
eroete and diaptay or\ engle hologram ae foHowo: 



void anole.histograitt { ras , threshold ) 

cip_edge_ro cults •res; /• results structure pointer •/ 

int threshold; /* significant edge threshold •/ 

( 

int i, j ,mag; 
int hist(256] ; 

cu.clear (hist, sizeof (hiatl ) ; 

for (1=0; l<ras->mag_ljngjp->wldtn; !♦♦) 

for (j=0; j<res->mag_linoj>->haight; j**) ( 

mag=re9">inag_lmg_p->get (res- proa g_iing_p , i , j ) ; 
if {mag > threchold) 

hist (res->ang_img_p->get (re9->ang_img_ p, j ) )^ 

) 

cgx_hi»togx»jn ( c»q„iinAga , hice . 651 ; 

caq^display ( 1 ; 
return; 

) 
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Figure 81 contains four snapes; each shape is paired with ds angle htstogrann. Noto that 
me square and the olus sign nave tdentlcal angle histograma. The angle histogram of the 
circle is flat because there is no dominant angle along the eogo of a cirdo. 




0 90 1B0 270 360 
Angto 




0 90 180 270 360 
Angle 




0 90 180 270 360 
Angle 



Ftgure 8i. Assortea snapes wnn tneir angle histograms 
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Figure 82 demonstrates tne eftea of rotauon on The angle histograms of a plus sign. The 
histogram shifts as the plus sign rotates. 
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Figure 82. Etfecx of rotation on an0e histogram 
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The Caliper Tool 

This section defines the Caiioer Tool ana descnbes in general terms how it works. The 
concQOts introducad in this sactton are aescnbad in dotail m lator cactions. 



The Purpose of the Caliper Tool 

The Caliper Tool is a tool for locating aages and edge pairs in an image. The eage or an 
object tn an image is a change in grey value from dark to light or light to dark. This change 
may opan cavoral pixfilc. TKo Cattp«r Tool proviooc mothocte for ignoring adg^a that aro 
caused by rvoise or that are not of interest lor a particular apotlcation. 

The Caliper Tool is modeled on the mechanical engineer's caliper, a precision aevice for 
measuring distances. You speafy tne separation between the caliper 'jaws.* and the 
Caliper Tool searches an area you specify for edge pairs separated by that distance. You 
can also search for individual edges wnen you kr>ow their approximate location in an 
image. This is similar to using a caliper to measure depth. 

A typioal ufio tor the Calipef Tool le pan inapection on an aosembty lino. For example, in 
integrated circuit lead inspection, a part is moved in front ot a camera to an approximately 
known location (the oxpBctBcs location). You usa tno Caliper Tool to oatarmine the axaci 
location of the left edge ot the part by searching for a single light to dark edge at the 
expected location (see Figure 1 03) . The edge that is closest to the ensected locauon is the 
left edge of the part. 
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Once you find the left edge of the part, you can oredici where the leads Ehould Ue within 
the rmage to specify a wmdCM/ m which to look tor leads (see Figure 1 04). You can then use 
the Caliper Tool to search for edge pairs consistina of a dark to light transition followed bv 
a light to dark transition, and separaiaa by the expected teed width. With information 
calculated by the Caliper Tool, you can compare measured lead width and location to 
expected lead width and location to make an accept/refect decision about the part 




FiguTB 104. The Caliper Toot can be used to measure lead width and 
load spacing on an tntegmted circuit. 

How the Caliper Tool Works 

The Caliper Tool uses projection lo map a iwo-aimensional window of an image (the ca//p©r 
wtndoW) into a one-aimensional image. Projection collapses an image by summing the 
pixels in tne dlreciion or me pro|eciion, wnicn tenas to ampiiry eages in mat direction. 
Figure 105 shows an image, a caliper window, the one-direcDona! image that results from 
projection, end a graph of the pixel grey values in The one-dimensional image. 
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First edge 
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Figure 105. Overview of the Caliper Tool 

You may specify thai the Caliper Tool oertorm pixot mapping before projoction. Pixrt 
mapping can be used to filter an image, attenuate or amolify a range of grey values, and 
othoAwoe map pixel values. Sao Choptor 3, Pixol Mapping, in the Imago Procwing 
manual for a complete descrtpiion of pixel mapping. 

After projection, an edge fitter is applied to the one-dimensiona! image to further enhance 
edge information and to smooth the image by eliminating minor grey value changes 
berween neighboring pixels that are most likely caused by noise. The edge fitter produces 
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a first derivative image (the most r aoid changes in grey value .n the proiecteo imaoe result 
in the highest peaks m the filtered image). Figure 106 shows the one-aimensional imaofl 
from Figure 105. an tmaoe Generated Dy me edge filter, ana a graph of that image 




On^dtmoneional 
intensity image 

One-dimensional 
edge filter imege 



Peaks in 

edge filter image 



First edge 



Second edge 



Figure we. An eage finer is uppti&a to tne one-atmensronai image that 
fGSutts from projection 

After the filter operation, edge detection is pertor mad on the filtered image. Edge detection 
15 a mfithod of ovaiuotirvg peaks .n the hlier ed wnage and ignoring edgee that era not of 
interest, such as eages repreaeniing nase m the image. Edge detection applies geometric 
constrmntsxnex you soecifv lo each edoe or adgo par found in the image. Applying 
geometric constraints provides a way to limit the number of edges to evaluate by assTgninQ 
a score to each eage or eage pair oased on such factors as the exoected location of the 
edge or edge pair, the distance between edges m an edge pair, and the minimum grey 
level difference between pixels on each side of the edge. 

The Caliper Tool returns as many results as you have specified in the cdp^arams data 
structure and computes the geometric average of the score of eacn geometric constraint 
that you specify. For edges or edge pairs whose score equals or exceeds the accept 
threahold. the Caliper Tool rotuma intormaiion such as the scort; and location of me eoge 
or edge pair and the measured distance t>eiween edges in an edge pair. 

The controlling variables of an edge or edge pair search define a caliper, and include such 
information qg the dimensions of the search window and the edge detection scoring 
parameters. The typa of tne data structure mat defines a caliper is cetp.callper. 
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The Caliper Window and Projection 

This section descnbes how you specify me caliper window, wnicn is me subsec of an image 
in which to tocate edpes or edge pairs. Ii mon describes oroiBcvon. which is tna mapping 
of me caliper window into a one-dimensional image. Finally it describes me edge fitter, 
which enhances edges in me one-dimensional imaoe. 



Caliper Window 

Tho eaiip^ window iq the poriion of an imagA in which tho Calipor Tool coarcnos for odgos. 
It IS defined by the search length and projection length, wnich are tne wioin and height 
resoectively. of the window in an image that witl be projected into a one-dimansional tmage. 
The caliper window is illustrated In Figure 107. 
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Figure 107. The Caliper Toot window 

The pixel in the ceraar ol me caliper window is called the appUcation point. This is the point 
in me run-time image at which you want to locate edges. 



Caliper Window Rotation 

For many applications edge information is needed at more than one angle in an image. For 
example, to inspect a rectangular pan, you may want to measure the dknensions of me part 
by oroiectino the image first at 0^ to measure length, and then at 90^ to measure width. The 
caliper window can be aiented at any angle to locate edges in an tnnage. Angles increase 
In me clockwise direction from the horizontal axis. 

Figure 108 shows how changing the caliper window angle affects the projection and edge 
search directions at 90' roQtions. The Caliper Toot runs faster tor 90* rotations man it ooes 
for arbitrary rotations. 
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Figure 108. The calipar window at 90"* rotations 



Skewed Wfndow Projection 

Figure 109 shows a -15*» caliper window with wjndow rotation disabted, tn this case, the 
Caliper Tool creates a "skewed" winaow ana projects it along the angle of skew into a one- 
dimensional image. 
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figure lOQ. Skmw^d window projocuon 

When window rotation is disabled, you specify how the skewed window will be projected: 
witn or witnout mtBrpolatton. Inierpolaiion increases accuracy at tne expense of execution 
speed. 

When you use a skewed window without interpolation for proiectior^. the pixels in the source 
image ore summed along the angle of skew as shown in figure 1 10. Each pixel in ihe 
two-dimenaonai image contributes to one pixel in the one-dimensional image. In 
Figure 1 10. thOM pixoic contQtning the number 1 contribute to the deeunation pixel 
containing tne number 1 . and so on. 
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Figure 110. Skewed window projecwn without tnrerpomtion 

When ycxj use a skewed window witTi interpolation for projection, tne skewed window is 
transformed into a rectangular window oetore profeaion. The transiormaiion is performea 
by clpjraneformO, which uses several neighdonng pixels from me two-dimensional 
image to caicUate the pixel value tor the one-dimensional image. See Chapter 2 Basic 
tmage Rrocessir)g, in the tmage Processlr^ manual for a complete aescnption of the 

tunotion otp.tran«forin(). 
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Rotated Window Projection 

Figur© 111 chowc a -16*' catip«r window with window rotation onat>ioa. In tnic cafto tno 
Caliper Tool roiaies tne two-dimensional image ana projects it into a one-dimensionaJ 
image. 




Choosing the Projection Type 

The choice of skewea or /-otaredprojecUon depends on the angle oi the edge or edge pair 
of interest m the image and the remaining content of the image. If the rotation angle is close 
to 90* or if a skewed image will contain enough of the edge or edge pair of interest to 
generate strong edges, you can use skewea profection to speed program execution. 
Figure 1 12 shows an image, a caliper window and the one-directional image that results 
from projection. The skewed window contains enough edge information, so skewed 
projection may be used. 
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Figure 1 12. Skewed projection may be used 

Figure 1 13a snows a caliper wincow wnere me eagas of interest are at 15» ana window 
rotation is disabled. A window targe enough to enclose these edges would also include 
much of the dark area in the imago, whicn in some cases would result in a one-dimonsional 
image where the edges are obscured. 

Figure 1 13b shows a caliper window at -IS"" in tne same image with window rotation 
enabled. Only the edges of interest are included in this window. 
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Figure 1 13. Skewea profection (a) ana rotated pro/ecvon (b) 



Caliper Tool Optimization 



When seeing up ttie vanaDles for a caliper, you soecify the following types of optimizauon: 

• Space optimaation: The Caliper Tod is optimized for minimum memory use at the 
expense of execution speed. 

• Speed optimtzBtion: The Caliper Tool will use a number of internal techniques to 
improve execution speed at the expense of maeasad memory use. 

• Default optimization: The Caliper Tool will weigh space and speed equally, using 
dtghtly moro cpaco than if you cpocify 6pac« optimization and slightly dow^r 
execution speed tnan if you specify speed optimization. 
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The Edge Filter 



After projection, an eage mens run on the resulting oneKaimensional image. The edge filter 
accentuates efloes in the image and produces a ttltarad imagQ. The paaks in this imacM 
indicate strong edges. 

The two parameters oi the edge filter are its size and leniency. Edge filter sue is the number 
of pixels to each side of the edge to consider in the evaluation. Edge filter leniency is the 
number of pixels at an edge to ignore between li^t and dark aides of the edge (lamency 
IS described in tunner detail below). Figure 1 14 illustrates an edge filter. The edge tilter is 
Doaitioned over the lettmosi pixel m the ono-dimonsionBt image where it will entirely fit in 
that image. Pixel values to the left of the leniency regon are summea and subtracted from 
m© fium of tho ptK»i veluee to tho right of the leniency region: in mta exomple. (0^6)-(0-0) 
yields S for the edge filter image. 
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Figufo 1 14. Tho oago mono poBittonod tn the one-dimenatonai imago. 



As shown in Figure 1 1 5. the edge filler is men applied at each successive pixel in the 
one<timensional image until the edge filter no longer fits entirely In the image. At each pixel 
position, the pixel values to the left of the leniency region are sunvned and subtracted from 
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the sum of the ptxet values to tne ngnt or me leniency region ana me result is written to the 
edge filter irnaga. The resating image will be 2 « size l«niency - ^ smaller than the 
one-dimensional imaoe. 
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Figure 1 15. The edge filter is appliea at each successive pixel until the 
edge filter r\o longer fits entirely in the image. 

Figure 116 shows a graph of the one-dimensional image and the edge filter image from 
Figure 115. PeaK position is catcuiaieo to suopixet accuracy. 
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imaoe 



Figure 1 16. Edge filter tmage graph vs. one-dimensional image graph 

The graph in Figure 116 rcpresontB an eage fiitar apolted to an laeat image- few 
applications would result in as ideal a graph. All Images contain some degree of noise 
a typical edge m an Image con be eevtsral pixda wide. The next two sections provtoe 
guidelines tot opiimizlng the edge filter by choosing appropnaie values for size and 
leniency. 



Choosing Edge Filter Size 

Edge filter size specifies me numoer or pixels on eitner side of an edge to consider in an 
edge evaluation. Size should usually oe greater than 1 because notse in an image causes 
grey level differences between neighPormg pixels. Figure 1 17 snows a graph of a 
one-dimensional image of a light band on a dark background, and a graph of an edge flitar 
image where a size of 1 was used. Because there rs noise in the image, many more peaks 
exist than are of interest. 
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Edge filter peaks 



Figure li7.An edge filter size of 1 is lypicaJly too small. 



Increasing size tenas ro smootn me eage finer tmage oecause summing several pixels on 
either side of an edge tends to reduce the effects of noise. Figure 118 shosws an image 
similar to the one in Figure 1 17 whoro a size of four was usoa. The only two peaKs in tne 
image are the peaks of interest. Size is typically between 2 and 5 inclusive. 
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Edge fitter peaks 



Figure 7 1 a Using four for edge mar size reduces the effects of naise on 
th^ Qdge litwr 

Choosing Edge Filter Leniency 

edge filter leniency specifies a region of pixels to ignore at an eage in an image. Due lo 
siigm changes in edge or edge pair position or orientation, a pixel on the edge of a teaiiire 
may not always have the same grey value. By setting a /e/7/encyzone between the light ar>d 
dark areas of a feature, the effects of slight changes in feature location on filter resuts are 
minimized. 

Figure 1 1 9 shows a graph of a one-dimensional image with two edges that span two pixels 
eacn. and graphs of the output ot two edge filters. The first edge filter has a leniency of 0 
the second edge filter has a leniency of 2. The edge fHter wim a leniency of 2 produces * 
higher peaks. Two is a typical value for leniency. 
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Figure 1 19. The effect of lentancy on en edge fitter image 
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Point Registration Tool 

This chapter descrtbes the Cognex Point Registration tool. It contains seven secoons. 

The first two sections, this one and Some Useful Definitions, provide an overview of the 
chapter and define some terms that you will encounter as you read. 

Point Registration Tool OvaA^/ew provides information about the capabilities and intended 
use or the Point Regtstration cool. 

Haw thB Paint RBgistrBtion Tool Works provides a oanerat descriDlion of the operation of 
the Point Regtsvation tool. 

Comparing the Point Registration Toot with Search compares the Point Registration tool 
with the Cognex Search tool. 

Using the Point Registration Toot describes some of the techniques that you will use to 
implement on application using the Point Regictration tool. 

Point Registration Tool Data Structures and Functions provides a detailed descrlpnon of the 
data structures and functions that you will use to implement your application. 
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Some Useful Definitions 

point ragistratlon A search technique designed to determine the exact point at 

which two images of the same scene are orecisety attorted. 

gblnl minimum The lowest value In a function or signal. 

local mtnfcna A low value in a function or signal. 




Local minima Global minimum 



eubpliel accuracy Positions within an image may be soecified in terms of whole 
pixal positions, in which caso the pocition refers to the upper* 
left corner of the pixel, or in terms of fractional pixel positions, in 
which case the oositlon may tie enywhere wtthin a pixel. 
Positions specified in terms of fractional pixel positions are 
referred to as having subpixel accuracy. 
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Point Registration Tool Overview 

The Cognex Poini Registration tool performs fast, accurate point registration using two 
images that you supoly. 

Point Registration 

Point registraiion is the process of determining the precise offset between two imaoes of 
the same scene. 

You use the Point Registration tool to perfomn point registration by providing a model image 
and a target image, along with a starting point within the target image. The model image 
snouid De a reference image of the feature you are anempting to align. The target image 
should be larger than the model image, and it should contain an Instance of the model 
Image within it 

The Point Registration tool will determine the precise location of the model Image within the 
target image. The Point Registration tool win return the location of this matcn. called the 
fogistration point with subpixal accuracy. 

Figure 160 illustrates an example of how the Point Registration tool performs point 
regtsuation. 




Model image 



Target image 



Location of model image 
within target image 



Figure 160. Point re^stration 
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The Point Regisrration tool is optimized to locate Drecisely me model tmage witnin the target 
image, even if me target image contains image defects such as refiections or if me modei 
image is partrally oDscured by otner features in the target image. Figure 161 illustrates an 
example of point registration where the target image is partially opscured 




Modol imoge Tarqet imoqe Location Of model image 

' within Ufget image 

figure 161. Point registration wit/) partially obscured target image 
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How the Point Registration Tool Works 

The Point flegistration tool finds Iho location of the model image within the taraet imaaa 
You operate the Point Registration tool by suoolying me model and target imoaae utno 
with the location within the target »nage where you expect the origin of the model imaqe to 
t^!; H f'^i^'^'^^on tool will determine, with sufipixel accuracy, where the origin of 
the model image lies within the target Image. 

The Point Registration tool works by compulir»g a score indicating the degree of similarity 

-!***"^,'"°''^' 'T^* ^"'^ ' t^'Set Image that is the same sae 

aa the model image. This score can range from 0 to 253, with a score of 0 Indicating that 
the fTiodel image and the target mage are perfectly similar and a score of 255 indicating 
met the rnodehmage and the target image are partectly dissimilar. The tool computes ms 
.? ' ^1°^^ ""Tiediate neighborhood surrounding the starting point. The tool 

will find the location wi.hin thie neignborhooa of the target image that produces ine local 
mininw in the value of this score. 

By adding an mtarpolanon step, the Point Registration tool then determines the location of 
me rnodel image within the target image with subpixel eccuracy. For typical images the 
Point Registration tool can achieve accurate registratoon to within 0.25 pixels. " 

Beaaue« of the way the Po.nt Registration tool ae«Ka the local minnna. if the starting point 
you specify is more than a few pixels from the actual registration point the tool may not 
return the correct regietration point. The sxaot amount of vahanoe that the Point 
Registration tool can tolerate will vary depending on the images. The variance may be as 
small as 3 to 5 pixels tor some Images or as targe as 30 pixels for others. 

Point Registration Score 

Each time it la mvokoa. the Point Rogistration tool will return, in addition to tne location of 
the ongin of the model image within the target image, a score indicating the degree of 
similarity between the model »r,«ga and the target image. T>,e acore returned by the tool 

m\ be from 0 to 256, with a score of 0 indicating that the model image and the target image 
are perfectly similar and a score of 255 indicating that the model image and the taraet 
image are per fecDy dissimilar. 

If your point registration receives a high score, the actual precision of the point registration 
location may be somewhat lower than if the point registration receives a low score. 

Exhaustive Point Registration 

When you use the Point Registration tool, you supply the tool with two images and a stanina 
location within the modal image. The tool will confine its point registration search to a small 
area around the starting location thai you specify. 
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The Point Regisirarion loot can also oerform point registration exnausiively. ttiai is, by 
computing t^e score for every possible location of the model image withm the target image. 
This procedure, called exhaustive point registration, is extremely slow. It can be helpful, 
however, in debugging applications where the Point Registration tool does not appear lo 
be working correctly. 

Masked Point Registration 

You can limit the areas of the model image and target image that the Point Registration tool 
uses to perform the point registration by supplying a senes of rectar>gJe3 to the tool. If you 
supply these rectangles, the too) will compute the score based only on Those pixels 
contained within the rectangles mat you speafy. 

Figure 1 62 illustrates an example ot specifying a masking rectangle, in inis example, the 
right side of the model is often obscured in the target image. By specifying a rectangle that 
oovflre thd left eide of the modd image, you c&n cause the Point Registration cool to 
consider only the pixels in that part of the mode! image. This will tend to increase the 
accuracy of the point registration. 




Masking 
rectangle 




Location of model image 
Model image Target image ^^^j^ j^^g^^ j^^g^ 

Figure 162. Masked point registration 

You can also specify several masking rectangles. Figure 1 63 illustrates an example where 
tho o«ntor of The modal imego rs often obaotirad in tho target image. By specifying a aeries 
of rectangles, you can eliminate from consiaeration the part of the tmage that is most 
frequently obacur^. 
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Masking 
rectangles 






Model image Target image Location of model image 

within target image 

Rguf0 7 63. M^skoa potnt r^gmration uatng muivpie roctangie^ 

"""^'^^'^ ""^^^^'"^ rectangles and the rectangles overiaD any oixels that ar^ 
contained ,n more than one masking rectangle wilt be counreo toward t^7sS mSe 
times, once for each rectangle in which they are contained. 



Image Normalization 



The Point Registration tool may not work well in cases where the model image and the 

SS^^fT^* r '"'^"""^ P«^°"" image processing as 

part of the point registration by supplying a pixel mapping table. 

It you Bupply a p.xel mapping table, every pixel ,n (he target image will be mapped to the 
ve^ua in the pixel mappirig table at the .oca«on v«th,n the Sixel mapping tTble gj^en S 
Pixel s value For examole. if a olxei ,n the taroet »n«ee had a value ofaoo it woUd be 
mapped to the value of the 200« element within the oixei mapping table 

ir you nave soecinea a masking rectangle, only those portions of the target image that lie 
within themastar^grectangleor rectangles will bemappefl through thepix^mapJfngS^^^^^^ 
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Comparing the Point Registration Tool with Search 

This section compares the Point Regisiracion cool with tna Search tool. For more aetailed 
intormaiion on the Search tool see Chapter i. Searching. 

The Point Regiairauon tool has me loltowing advantages over the Search tool: 

• It Is less sensitive to images that contain areas ot pixels that have widely different 
values than the same pixels in the nrxxaet image (i.e.. blotches or occlusions). 

• It requires no training step. 

The Point Registration tool has the following disadvantages when comoared with the 
Search tool: 

• It ts capable of tine alignment only. 

• It does not work with scaled or rotated target images. 

• It does not work well with images with brightness changes. 

• It will not find or rank multiple regtsuatton ooinis. 
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Using the Point Registration Tool 

This section describes how to use the Point Registration tool. 

Obtaining Model and Target Images 

You acquire the tn^ages thai you use as model and target images uang the techniques 
described in Chapter h 3000/4000 Acquisition and Display, in the 3000/4000 Image 
Acquisition and //O manual and Chapter 1, 5000 Acquisition and Display, In the 5000 
imsg0 Acquisition ond I/O manual. 

You should acquire all images that you plan to use with the Point Regisuatton tool, 
especially images of scenes with low contrast, using a bit depth of at least 8. For fastest 
performance, you should acquire images as FASTS images. 

When you acquire the model image, you should take care to ensure that the image is as 
ideal a model as possible of the images that you will be using as target images. The Point 
Registration tool is designed to align target images with a wide variety of image defects, 
but in ordor tor the tooi to work, th« modof imago neoda to bo ae free from defoots as 
possible. 



Specifying a Starting Offset 

When using the Point Registration tool, you must specify a starting location in the model 
image. The too) will perform its point registration search starting at the point that you 
specify. Because the tool only seeks the local minima for the score value, If you specify a 
starting location that varied graetly from the actual registration point, the point registration 
operation will fail. 

You can use other vision tools such as the Search tool or tne inspection tool to locate the 
model feature within the target image, or you can rely on operator input to specify the 
coarse location of the feature. 
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Point Registration Tool Data Structures and 
Functions 

This aeciion coniains descripoons ot the Point Registration tool data siruciures and 
functions. AH Point Registration tool data structure and function names begin witn tne orefix 
cr©g_ 

Structure creg.params 

cr«g_p«nima contains parameters that determine how the point registration search wilt be 
conducted. A structure of this type is suopllea as an argument to creg_pokit.register(). 

Ilnclude <regi8ter.h> 

cypedef struct 
( 

c_Int32 3tart_x. 

c.Int32 recc_count ; 
ch«r *optional_/napf 
c_Iut32 flags; 
} crea^&rams : 

• stsrtx and stan^y are the x- and y-coordinates of the staning Dosmon (or the point 
registration search. 

• recLCOL/nr is the number of reciangles in me racts argument to crao-PolnO'eoleterO. 

• optionaLmap is an optional pixel map. If opuonaijnap is non-NULL, the Point 
Registration tool wiii replace each pixel in the target image with the value contained in 
the element of optional _fnap corresponding to the value of the pixel in the target 
imago. H you cuppiy a value for optionsi_map. you nr>uat supply an image of type 
FASTe. 

You can use optionaLmap to compensate lor image-to-image brightness variations. 

• flagsspedfies tne type of point registration mat me Point Registrauon tool will perform. 
flagsls constructed by ORing together any of the following values: 

• OREGjyPE 1 is reserved for future use; you should afways include CREG^TYPEI 
in tne value of flags. 

• CREG_TYPE2 IS reserved for future use; you should never include CREG_7YPE2 
In the value of flags. 

• CREG.NORMAL is useo to specify a point registration based on finding tne local 
minima of the score value. 
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• CREGJEXHAUSJiyE\s used to specify a point registration based on finding the 
global mintmum or me score value. 

If you specify CREG_NORMAL and if stan^x and sfarO' are more than a few pixels 
. from the actual registration point, the tool may return a location that does not represent 
tne Desi matcn wiinin tne image, it you specify CREG^EXHAUSTIVE, the registration 
will return the beat registration march tor the entire imago. An exhaustive point 
registration will be extremely stow. 

Structure creg^results 

cretLPomij-^glster returns a pointer to a cregLretutU structure, creg jM|m.r«giater 

fills in the structure with information about the results of a point registration operation. 

♦include <rogister.h> 

typedef etruct 

( 

float X, 

y; 

c_Int32 ccoro, 

time ; 
char on_edge_x. 
on^edge^; 
) creg.ra6ulC£; 

• rand y are the x-coordinate arvi y-coordinate, respectively, of the location within the 
model image at wnicn the target image was found, x and y give the position with 
$ubpixol su:curacy. 

• scofB is a measure of the similarity between the pixel values in tne model image and 
tne target image at ine nearest whole pixel alignment position, score will be between 
0 ana 255. with lower values indicating a greater degree of similarity between the 
model image ana the target Jmage. 

• time is the total amount of lime that trte tool spent on this point registration, in 
milliseconds. 

• OtL^dgo^x is sal to a noniero value if, along the x-axic, ono edge of the model image 
area is at the edge of the target image. If on^edge^ is nonzero, the accuracy of the 
position information may be sliohtty reduced from what it wouid otherwise be. 

• or\^edgB^\% sot io a nonzero value if. along the y-axis, one edge of the model image 
area ta at the edge of the target image. If on^edge^ Is r>onzero, The accuracy of the 
position information may be slightly reduced from what it would otherwise be. 
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Function creg^poinLregisterO 

crAO_polnt_peol<t*r() porlorms point rogtetration ucing th« rnodol image and largat image 
ttiat you supply. The point registration be controlled by the parameters in the 
crag parama structure suootied to ine tuncDon. croo_i»olnt.r*glctar() rstums a oointer to 
a crog.r6sijlts structure that describes the result of the point registration. 

tinclude <re9ister.h> 

creg^resuits •crag_poinc_regiocer ( consc cip_buffer 'target . 
const cip_buffer •model, const cia_rect *reccs. 
const crag_parainB "paraine, creg_re2uits •results); 

• targot points to the image to use as the urgat rmago for thic point ragictration. target 
muse be at least one pixel larger in botn the x-dimenson and tne y-dimension than 
model. 

• /notfe/ points ro the image to use as the model image for this point registration, modal 
must be at least one pixel smaller in both the x-dimension and the y-dimension than 
target 

• rects points to an array of cla_rect structures. If reels is non-NULL. the Point 
Registration tool wit) limit tne comparison of pixel values to just those pixels that lie 
wuhin tne reaangies coniainea in rects. If tne rectangles in recfs overlap, a pixel will 
be included in the comparison once for each rectangle that contains it. 

• params points to a cregjMrams structure that defines the parameters to use for this 
point regisuailon. 

• results points to a creg_reeuft structure. creg.polnt.reglster() will place the results 
of this point registration in rmsults and will return a pointor to /oeultc 

if you supply NULL for results, crea_potnl_r«gi«tar() will allocate a crag_result 
structure from the neap using the default allocator. You can free this structure by 
celling tree(). 

creaj)olnt_rcolgtor() throws CGEN_£RR_BADARG if 

• rects is NULL and the recccount field of params is not 0 

• If model 'xs larger than or the seme size as target 

• If the sfarr^x field In params is less than 0 or greater than the width of target rvimus the 
width of model 

• If the slflH-V field In oarams is less than 0 or greater than the height of fargef minus the 
height of model 

• If moc/e/ is NULL 

• If params is NULL 
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• If fa/ger is NULL 
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3 Line Finder 



An Overview of the Line Finder 



The Line Finder computes Die locauons of lines in an image. For each line thai it finds it 
r«turn6 an angte e and a signed disunca dfrom th« canual pjx«l of m« imaga. GiVAn e and 
d. you can compute ail points (x.y) in the image that satisfy the equation of the line (as 
illustrated In the section The Trsnsformsnon from Canesian Soace to Hough Space on 
page 204). The algorittvn that the Line Finder uses, along witn the angie/distance 
classification of lines, is explained in How the Line Finder Works on page 166. 

Vou can use the Une Finder in either of two modes, quick or normal. In quick mode the Line 
Finaer provtaes me locaiion or me poini along eacn une mat \s nearest to me center of tne 
image, along with the angle of the line. In normal mode, the Line Finder also estimates the 
length and donaty of each Una tnat it locataa. uoing statistical methods, and raturna ma 
position of the center of mass of the line. 

Figure 78 illustrates the two modes of the Une Finder, The image in Figure 78a Is shown in 
Figure 78b after the Une Finder rms been invoked in quick mode. The presence of the four 
lines is indicated. Each line is drawn to the screen at a constant length, but no anempi is 
made to estimate its actual length. Figure 78c shows the result when using the Line Finder 
in normal mode: the length of each line segment is estimated. 

For a aetaiiea explanation or me aaia reiurnea oy tne Line Finaer m eacn moae. see ine 
section Resufts in Quick Moae and Normal Mode. 



Figure To. An image (a), tines rouna in quicK moae (oj. ana line 
segments found in normal mode (c) 

The Line Finder finds lines regardless of grey scale variations In the image, rotation of the 
scone, or changes in scale. A tyoical apolicaiion tor the Une Rnder is locating fiducial 
marks in a series of images In which grey levels, scale, and rotation are unpredictable from 
one image to the next. Figure 79a and Figure 79b illustrate two images, each containing 
the same fiducial mark: a square within a square. However, the sizes of the marks, and their 
rotations differ between the two tmages. Also, the mark in Figure 79a is a dark square wimin 
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a lighter square against an even lighter background, but the mark in Figure 79b is a light 
square withtn a darker square against a darker background. As shown in Figure 79c and 
FiQure 79d. the Line Finder (tnds the outlines of the tiduciat marks oesoite these variations. 




c d 



Figure 79. The Line Finder is immune to grey scale, size and rotation 
changes 

Since tne Line Finaer is an Intermeaiaie-ievei tool, your appiicauon win prooaoiy require 
that you do further processing, using the results of the Line Finder as input. You will 
probably not be searching tor lines in your image. Out rather for a teatLtre whose presence 
ts implied by the locations of lines. For example, in Figure 79. the Line Finder does nor 
return iho loceiiona arjd dimeneione ot the fiducial marke, only the looatona of the lines that 
bound the nested squares. Computing the precise locations and sizes of the fiducial marks 
requires post- processing based on the Line Finder's results. 

The Line Finder demo code includes post*processing of this sort. It contains functions that 
locate squares ot any size and rotation, using the Line Finder. This code deduces the 
presence of squares based on the Line Finder's results, eliminating extraneous lines. See 
Chapter 1, CognexAPi Introduction, in the Development Environment 1 manual for the 
location ana name of the Coniou Finding demo code on your system. 
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3 Line Finder 



How the Line Finder Works 



This chapter explains hew the Line Finder finds tinea in an image. This discussion is divided 
into the following cubGdctionG: 

• DBfining a Line by Angle and Distance describes the definition of a line used by the 
Line Finder. 

• Hougn Space introduces the two-dimensional space in wnich the Line Finder records 
lirms. 

• The Hough Line Transform describes the algorithm trtat the Line Finder uses to find 
lines In an Image. 

The Line Finder uses a Canesian coordinate system whose origin is the center of the image. 
Most of the examples in the book are drawn in this coordinate system, although a few use 
image coordinates. These two systems are pictured in Figure 80. Notice that the x-axes of 
the two systems are the same, but the y-axes are opposite. Note also that positive angles 
in Cartesian coordinates are measured counterclockwise from the x-axis; positive angles 
In image buffer coordinates are measured clockwise from the x-axIs. 



-y 



Image buffer 






Cartesian 


coordinates 






coordinates 






J- 


-X 




+6 



+y -y 

Figure 60. Image buffer coordinates and Cartesian coordinates 

The Line Finder uses the central pixel of the image as a reference point. The image buffer 
coordinates of this pixel (x^. yjare computed with integer division, using the following 
formulas: 
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wnere wis tne wiain of tne image ana Mis tne neignt ot the image, in pixels, tn the Line 
Finder, the Cartesian coordinates of the central point in the tmage are (0.0). 

Defining a Line by Angle and Distance 

The Line Finder defines a line (in Canesian coordinates) by its angle from the x-axis and by 
the signed distance from the center of the image (the cervral pixel) to the line. Fnese two 
parameters uniquely describe any lir>e in Cartesian space. 

The feature in Figure 81a contains an edge that lies along a line (Figure 81b). The line is 
defined by its angle from the horizontal (e) and by the shortest distance from the center of 
me image to tne line (Figure 8i c). Note that the distance vector is perpendicular to the line. 




Figure di. Angle ana distance oeftne a line 

In the definition of a line by angle and distance, the distance is rtegative if the angle of the 
distance vector (in Cartesian coordinates) is greater than or equal to 0** and less than 1 80^: 
the distance is positive if the distance vector has an angle greater than or equal to 1 80** and 
less than 360^, 

Distance is signed to avoid the ambiguity illustrated in Figure 82. Figure 82a shows an 
image wiin two features. Each Teatur e is Doroerea on one side by a line. Aitnougn tne lines 
are different, they have the same angle e and the same absolute distance dfrom the center 
(Figure e2b). 
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Figure 82. The angle of the distance vector determtnes the sign of the 
dietstnce 

Figure 82d) each line is ensured a unique definition. In this examDie, the two lines are 
denned as (9, cJ) and (9, -d). 

Once you know the distance and the angle of a line, you can oomoute ail points (k^ y,) ,n 
Certeaian ccordmates, that l.e on the line. Given a distance, d and an anje 6 a fne^s aH 
points rjf»y,; that satisfy the tallowing equation: .""neisaii 



d = Xotine-ypcoie 



The derivation of thio formula m auppliod in th« .aotion Tho TranGtormaUon from Can^dan 
Space 10 Hough Space on page 204. As an example of its application, if e is 4S and dis 0 
a line is all points in Cartesian space that satisfy the following equalities; 
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0 = xsin(45)- ycos(45) 
X = y 

Hough Space 

Mcxigh space is a two-dimensional space m which ihe Line Finder records me lines thai it 
finds in an image. A point in Hougn space represents a line; one axis represents angle, the 
other represents distance. 

Hough space is implemented in the Line Finder as an array: a simple example is shown in 
Figure 83. Tho Hough spaco in this oxomplo can rocord Gtgm anglac and nine diatancea 
and therefore 72 lines. 

The Line Finder lets you control the size of Hougn space. You specify (he rarige o1 distance 
in pixels, and, in the edge detection data structure that you pass to the Line Finder, you 
supply the number of angles that the Line Finder can ccmpuie. 



315 

270 

225 

Angle 

135 

90 

45 

0 1 I I I I I I I I 

.4-3-2-10 1 2 3 4 
Distance 

Figure 83. Hough space 

The Hough space in Figure 64 contains a single line. It has an angle of ^ZS'' and is a 
distance of 3 units from the center of the image. 
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imago 



figure 64. Hough space conisming a single Une 



The Hough Line Transform 



The Une Finder uses me Hough Une Transform to locate lines in an image This method is 

implemented by this tool, takes advantage of the fc^lowmg 
ule. i( an eoge pue. s angle , ana location (;rO,yO) ,5 known, men the line on which me p xel 

rjr^^^^^^^ ""^ '^^'^ — Pixel (.O.yO,%^h-s" "'^^ 
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b c 



Figure 85 Four eages in an tmage (a), eage angtas at four eaga pixels 
(b). ana line angles ai those pixels (c) 

The Hough Line Transform is implennenied as follows: 

1 . Using edge detection. \t\e Line Finder creates an edge magnitude image and an edge 
angle image from tne input image. 

2. The Line Finder creates and clears a Hough space. 

3. For each edge pixel m the input image, the Line Finder does the following: 

• Using the edge angle ^ and location (xO.yO) oT the pixel, itie Line Finder calculates 
the line's distance from the center, d, and its angle, e (using the tonmula 8 = o t 
90). 

• The Une Finder increments the bin in Hough soace reDreseniing the line (O.cO. 

4. Once alt edge pixels in the image have been examined, the Line Finder searches 
Hougn space for maximum values, using peak aeiectlon. The highest value in Mough 
apace represents the strongest line in me image. The second highest represents the 
second strongest line, ana so forth. 

5. The location ol each line is primed out and, optionally, each tine is drawn in an output 
image. 

Figure 86 through Figure 89 illustrate this method. 

Figure Bfia is an tmage passed lo the Une Finder. Figure aOb is a stylized edge 
magnitude image created from the input image. 
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Figure 66. An input image (a) and the edge detected input image (b) 

Each edge pixel is processed, as described above. For example, because me oudined 
edge pixel in Figure 87 belongs to the *ino (90. -2). the bin in Hough space tnar represents 
that line is incremented. 
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Figure 67. The bin in Hough soBce for the edge pixel's line is 
incremented. 
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Figure 88 shows me final Hough space atong with tne input image. Figure 89 shows the 
peak-detected Hough space, with each c1 tne peaks outlined. Figure 69 also shows tne 
input Image with the lines, represented by the peaks in Hough space, drawn into the input 
inriage. 
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Figure 68. The final Hough space and the input mags 
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Figurv 69. The peak-detected hough space ana the input Image, with 
lines added 

Notice that the Hough space coniains many extraneous lines. If. for instance, tne vision 
problem IS to find the tfiangle in the image, the lower scoring lines that border the blob need 
to be eliminated. Almost all of the Line Finder applications require posi-processing of this 
sort. In this example, simply locating the three highest oeaKs in Hough space eliminates 
the unwanted lines (Figure 90). In most applications, the required post-processing is more 
complex. 
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Figure 90. The output image after post-processing 
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APPENDIX II 



Patent Application for 



10 



15 



MACHINE VISION CALIBRATION TARGETS AND METHODS 
OF DETERMINING THEIR LOCATION AND ORIENTATION 



Software Listings 



THE FOLLOWING APPENDIX IS NOT BELIEVED TO BE NECESSARY FOR 
25 ENABLEMENT OR BEST MODE DISCLOSURE OF THE INVENTION DISCLOSED AND 

CLAIMED IN THE ACCOMPANYING APPLICATION. 
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Copyright (c) 1996 by Cognex Corporation. Natick. MA 'JSA 
All rights reserved. This material contair.s ur.pubiished. 
copyrighted work, which includes ccnfidentiai anc prcprieta: 
information of Cognex. 

cwa_f ind_target . c, Aaron Wallack - 13 Apr 1996: Created. 
Time-stamp: <96/07/30 09:32:47 awallack>: last modified. 

SRevision: /main/noneS 



(^include 


<prealign . h> 


**include 


<math.h> 


itinclude 


<stdio . h> 


# include 


<cct .h> 


^include 


<cip.h> 


J^include 


<ctm. h> 


^include 


<caq/caq_def . h> 


S include 


<cwa_priv . h> 


include 


<search/ctr_def 


^include 


<search/cse_def 


n include 


<clp/cclp.h> 


itmclude 


<clp/ cclp_er . h> 


it include 


<chp . h> 



This file includes a f-^.cti: 
square) target. 



a crrssr.air 



The algorithm is intended to handle the cases when the target is 
imperfect (occluded or broken) 

i*^#i*#^^ij#^^#j^#i*##^itij^*^4t##i*i^^iif (occlusion) 
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broken 

############### 

♦ These routines are intended to be used as a subroutine for 

♦ determining the cameras' field of views (wich respect to an absolute 

* coordinate system defined by a motion stage) . 

♦ The target localization routine {cwa_f ind^target ) takes as input: 

♦ img - a cip_buffer containing an image of the target 

♦ calib - a calibration structure intended to characterize the 

* non- rectangular ity of the pixels 

• angle - the orientation of the crosshair ir. terms of ixace 

* coordinates {we require this 

* orientation estimate to be accurate -o 3 degrees. :.?5 racia 

* result - pointer to a cwa_lir.e structure where the result will be 

* returned 



* Requirements ; 

* Image quality: 

* The image of the crosshair must be visible and perfect 

* (without occlusion or breaks; throughout a I 00 X IOC pixel square 

* surrounding the crosshair center. 

* Consequently, the center of the crosshair T.ust be at least 

* 50 pixels away from the borders of the cip_ouffer. 

* Orientation estimate accuracy: 

* The given angle estimate {given ir. terms of image coordinates) must be 

* accurate to withm 3 degrees 0.05 radians 



* This algoritm tolerates occlusion and breaks in the target 

* so long as these imperfections occur at least 50 

* pixels away from the crosshair center 



* Major Consideration: 

* One interesting thing to note about crosshair calibration, and the reason 

* that we use four distinct rays, is that the separation between the opposite 

* rays depends upon the lighting and aperture of the lens. Furthermore, we 

* cannot assume that the separation between these edges will remain constant 

* from one setup to another 

* What we really want is the position of the 
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♦ Since we do not know whac zhe of f set / separacicn is, it turns out that the 

♦ repeatability of the entire fit ( localization of the trosshair) is bounced 

♦ by the accuracy cf the smallest of these ftur lines. It dcesn't make any 

♦ sense to include 10000 edge points frcm cne ray if we only have 50 edge 

♦ points from another (the localization error will still be on the order of 

♦ 0.1/sqrt(50? pels, assuming randcr. i.i.d. errrr ano 3urpi;:el Itcalizaticn 

♦ accuracy of 0.1 pixel) . 

♦ We want to inccrpcrate the same number tf edge t tint 5 frtr. zppcsite rays 

♦ because we always want tt fit the lines tt the same e::act points tor 

♦ highest accuracy and repeatability' . when the talibratitn target is 

♦ occluded by a particular object (such as the borders that can be seen m 

♦ the database- , then if we use the exact same number of pixels on the 

♦ opposite unoccluded ray as the number of pixels on the occluded ray. then 

♦ we will always be locking at the sane pixels on the -unoccluded ray 

♦ Basically, since we do not know which edge points 

♦ imoer feet ions in the target and whizh eoge pcmts 

♦ valid points on the target, we use a boot-strappi 

♦ compute regions where edge points are VAI.I-. 

♦ Assuming that the target is perfect wizhm the GUAHANTEZT GOOD 

♦ REGION (center -^Z- 50 pixels;, we can com.pute four GOOD SAVS 

♦ characterising the four rays emmating from the crosshair center. 

♦ Then, we can use these GOOD ^YS tt define the VALID REGIONS (the 

♦ set of points within a TIGHT_THRESKCLD of the GOOD RAYS) . Finally, 

♦ we im.prove the estimates of the four rays by going back and 

♦ enumerating edge points along the GOOD RAYS until we encounter an 

♦ edge point which falls outside the VALID REGIONS. 

♦ Definitions 

♦ GUARANTEED GOOD REGION 

♦ region surrounding the origin where we insist that the calibration target 

♦ must be perfect 

♦ GOOD RAY 

♦ A ray fit to points enumerated wiihin the guaranteed good region 

♦ VALID REGION 



correspond to 
ccrrespond to 
ng algorithm, to 
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Region surounding good ray where we crust chat the edge points are due to 
Che crosshair and not from occlusion or breaks 

OUTLIER POINTS 

Pomes which we cannot crust because they do noc fall withm the valid 
region 

CENTER OF CROSSHAIR TARGET 

=u---er 
edge points 

^^^m^:^^^ < good ray 

ni^^^^i^^^ VALID REGION 



^ .11.4114 ^11.44 a 



-guaranteed gccd regicn 



lanci. ; 



* We are adopting a better- safe- than-scrry stratec-.- z 

* occlusion (as soon as we see a ■'bad" pcmt , we Icse fai" 

* the siibsequent points) . Although tnis approach is suboc: 

* database images seem to contain nearly perfect ooints emmat: 

* frcm the origin that it should work well in practice. 

* Algorithm: 

* Use absolute ncr:nalited tcrrelation searoh to apprcxinatelv 
locate the crosshair center 

* This involves constructing a synthetic T.odel of a rotated 
crosshair center (see gross__lccace_trosshair_center m in-iage ■ : . 

2) Given this initial position estimate and the user -supplied orientation 

* escimace, improve the positional and orientational estimates by localizing 

* four good rays eminatmg from the crosshair center. This involves 

* enumerating edge points within the GUARANTEED GOOD REGION (50 pixels) 

* along che supplied oriencation as well as 

* theta+PI/2,cheta-t-?I, theca + 3*?I/2. These compuced four GOOD RAYS are used 

* in step 3 Co check whecher edge points are outliers. In other words, these 

* four good rays define VALID REGIONS separating good edge pomes (poincs we 

* believe correspond Co che crosshair) from bad edge points' (points we 

* believe are due to occlusion or breaks) We should also point out that we 

* fix the orientations of the four rays by "averaging" these orientations,. 

* i.e., we incorporate the constraint that the four rays correspond to 

* orthogonal physical lines. (see 

* cotTTpute_center_j)osition_and_angle_assuming_four_guaranteed_good rays ( ) ) 

* 3) Go back and enmerate edge points along the four rays -until we find an 

* edge peine which does not satisfy the VALID REGION constraints. We also 

* throw out some of the edge points in order to realize equal numbers of 

* edge points from opposite rays. (see 
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* compuce_cencer_posit:icn_anci_angle_aicng_guarar.ceed_5ccc_ra*/s ; ' 

♦ NOTES 



♦ When we enumerace edge poincs along rays eminacmg frcrr. z'r.e crigir. . 

♦ we start out a small distance away from the origin sc zhac we den ' t 

♦ get confused by two different edges (this buffer zone is 

♦ characterized by the constant CIRCrJI-AR_BUFFERj2CNE_ARCrNr_rz^rrE?.. 

♦ which is 10 pixels) 

♦ The central function in this algorithm is 

♦ enumerate_points_and_f it_line_along_ray { ) which enumerates edge pcints 

♦ along a specified path, enumerate jpoints_and_f it_line_aiong_ray • ) applies 

♦ calipers at sample points along the path (the edge estimates can be 

♦ further improved using parabolic subpixel interpolaticn; Along a specific 

♦ path, enumerate_points_and_f it_line_along_r3y ) always makes integral ur.it 

♦ steps in either the x or y direction. This is sc that we can apply 

♦ axis -aligned calipers along every row or column when the step site is 1 

♦ enumerate_pcints_and_f i t_line_altng_r3y . : enumerates ecge ptint3 

♦ until It either encounters a bad edge point, or finishes sar.p-inz 

♦ the edge segment. At this point, it ceases enumerating edge pcints. 

♦ and fits a line to the pcints. There are four ways an edge pcmt 

♦ can be ''bad" . If enumerate_pcints_and_f it_line_along_ray : : 

♦ encounters any of these four situations, it returns C after fitting 

♦ the line; if it successfully enumerated all of the edge points 

♦ along its specified course, it returns 1. 

1. no edge is found by the caliper applied at the sample point 

2. more than one edge is fo'und by the oaliper and the highest 
scoring edge is less than twice the score of the second highest 
scoring edge (this 2X comes from the CCNrUS ICN _THRZS HC ; 

3. the computed edge point is an outlier tthe distance between the 
edge point and the nominal line is larger than threshcld) 

4. the caliper extends outside of the cip_buffer 

♦ When we call enumerate_points_and_f it_line_along_ray { ) in step 1. we 

♦ specify a ray from a point near the crosshair center to a point 50 

♦ pixels away from the crosshair center. We do look at the return val 

♦ to check that the enumerate jpoint3_and_f it_line_along_ray () did not 

♦ encounter any problems . 

♦ When we call enumerate_points_and_f it_line_along_ray w in seep 2, we 

♦ specify a ray from a point close to the crosshair center along a 

♦ good ray to a point 1000 pixels away (guaranteed to be outside the 

♦ cip buffer) . Thereby, even if we do not see any "bad" points, we 

♦ will undoubtedly leave the cip_buffer at some point. 

♦ The reason we pass a calibration object to cwa_f ind_target is that we 

♦ recompute the orientations of the four rays by incorporating the Jcnow ledge 
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chac they all ccme from orrhogcnal lines. If zhe pixels ar- rc"-scua-- 
(I.e., by '/ircue of zhe fact chac the CCD elements are ncn-squaro 
we need to account for this when we "average" the fcur crientacc-s We" 
"average" the four orientations by transforaing the'Image 'orlentat^-^nr 
into physical coordinates (and then truly averaging me ohvsical ang^-s 
and then transforming the average angle back into image coordinates ""we 
cry to use the term "phys" to denote physical coordinates and "img" 
denote image coordinates {such as phys^orient and inc_trient) . 

We use different sized calipers for steps 2 i 3 because we make 
dxfferent assumptions about the relative accuracies of our caUpe- 
application points. We use the terms LOOSE and TIGHT to refer -o -he 
calipers; LOOSE for step 2 where we have little faich -n^-he 
application points, and are thereby forced to*use broad^dlipe-s a-c 
TIGHT for step 3, where we have much more faich in che atplicat-cn 
points, and can thereby use relatively tight calipers. 

we expect the initial search position lo be accurate -c : oe^s an-- 
Che orientation estimate to be accurate to 0.35 radians. At a ca-'ne- 
applicaticn point 5C pels away, the talioer x.av be c-'" cv "d 
pels from the correct edge pcsiricn. Just tc oe saf " vl'-^s- T 
caliper with prcrecticn length 24 :i: cn earn sice; r: enu:::erate -za^ 
points in stec Z 



* After improving the position and ^rientaticn estiT.a-e- m stec Z 

* basically assume that we've get the ri-ht ccsiticn zz - - o " oo's 

* and */- 0.01 radians. Thereby, we can use these lines -c threshold 

* outliers because we would expect errors of onlv 2.2 eels Iz'zoT'^ 

* pixels away. This is pretty tighc because we use a ihreshcld of 3 . 0 

* pels to detemme outliers 

* (actually, the C.2 pels, CO: radians could s ignif irantlv moact -he 

* perrormance because the firsr 50 pels and the oriencarisn estima-« 

* end up predominantly affecting whith edge pcmts are included m the 

* final line estimates; 

* Main subroutines - 

* gross_locate_crosshair_center_in_image '.; 

* construct a synthetic model of the crosshair and use search to 

* estimate the crosshair position 

* f ^C_line_to_points ( } 

* fit a line (x,y,t; to an array of points 

* compute_subpixel_position { ) 

* use parabolic interpolation at the neighborhood of the application point 

* to compute the subpixel edge position 

* enumerate_points_and_fit_line_along_ray() enumerate edge ooints along a 

* line segment from start_point to end_poinc and possibly store the found 
edge points in an array. Furthermcre. a nominal line and a threshcld 7re 

* passed in as arguments, and when che found edge pcmcs deviate rrom'th^s' 

* nominal line by more than the threshold, enumeration is stopped'and T'lme 
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• is fit CO the enumerated edge positions. Enumeration is also stonoed 

• two edges are found (confusion) , no edges are found (cctlus ion; or wh) 

• the caliper is applied out of bounds (out of bounds; 



* average_f our_image_orientations i ) 

* average the orientations of the four rays to compute the best 

* composite orientation estimate. This function transfor:r.s the irnage 

* orientations into the physical domain in order to explcit the 

* constraint that the physical lines are orthogonal 

* compute_center_position_and_angle_assuming_four_guaranteed_good rays () 

* improve estimate of the crosshair position and orientation by 

* searching four regions of guaranteed good length (50 pixels) 

* eminating from the crosshair center. Lines are fit to the edge 

* points along the four regions, and then the orientations of these 

* lines are "averaged" . Two orthogonal lines are computed by 

* averaging pairs of the four rays Finally, the position esti:nate is 

* computed by intersecting the two orthogonal lines 

* compute_center_pcsiticn_and_an9ie_alcng_5uaranteec 

* final estimate cf crosshair pcsiticn and orientation 

* along four rays (using four gccd rays as thresholdm 

* tighter calipers (than we -^sec m step 2. because we 

* of the good ravs . 
*/ 



cod_rays ; . generate 

by enuT.eratmr points 
^ lines . We use 
rr-3t the positions 



^define N_Er;G£S 10 

^define GUAXAKTZz:D_GCCD_SIZE 50 

i^define GR0SS_TARGET_SI2E GUAIIANTEE2_GC0D_SI2E 
^define NUM_S AM PLE_ POINTS 1000 
#define LOOS E_THHES HOLD 12. 

^define tight^threshcl:; 3. 

i^define L00SE_CALr?HR_WI3TH 24 

/* ( (int) 2* ;L0CSE_TKR£SHGLD*GUARA:rrEE3_GCCD_SIZE*sin (0 . :5i : - 2 (kernel)- 

^define TIGHT_CALrPHR_WIDTH 14 

/* ( (int) 3*TIGHT_rHR£SKCLD) * 2 (kernel;*. 

/♦ 

♦ TIGKT_CALI?ER_V7IDTH should be larger than the minimum width so that 

♦ we check whether there is a sharp edge just outside the 

♦ TIGHT_TKRESHOLD rangs, because if there is a sharp edge just 

♦ outside the T;GHT_THjI£SHOLD range, then we want to declare 

♦ confusion and stop enumerating ooints 
*/ 



i^define L0CSE_CALIPER_PROJECTION_LENGTH 5 

#define TIGHT_CALIPER_PROJECTI0N_LENGTH 5 

^define MAX_NU«_POINTS 1000 

itdefine EPSILON_THR£SHOLD 0,05 

#def ine CIRCULAR_aUFFER_20NE_AROUND_CENTER (8 . +LOOSE_CALI?ER_?ROJi:CTICN_LENGTH) 

^define STE?_SIZE_USED_TO_ENL^RATE_POINTS_IN_GOOD_REGICM 3 

^define STEP_S 1 2E_aS ED_TC_ENLT<E RATE _ALL_ POINTS 1 

*tdefine MAX_LENGTH _D I AGONAL_ACROSS_CIP_ BUFFER 1000. 

i^def ine CONFUSION THRESHOLD 3 . 
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/♦ number of pixels to igr.cre when we mrc an cucl-er ' 

(^define OUTLIER^BUTFER 7 

excem int cd_3howas ( inc . char*); 

excem double cwajpoinc_distance ( ) ; 

char* cy_prealigTi_cwa_f ind_cargec 1 ) ( return rllE ; 

static int cwa_f ind_target_debug__f lag=0 ; 
int 3et_cwa_f ind_target_debug ( int x) 

{ 

int old_flag=cwa_f ind_target_debug_f lag; 
c wa_ f ind_ t ar ge t _debug_ f 1 ag = x ; 
return old_flag; 

} 

typedef enum calib_orient 
{ 

CAI-IB_HORIZ=0, 
CALI3J/ERT 
} cal:-b_orient ; 

/* A training parameters record for training related rrcssr.j 



sta 


W A w 


ct 


r_par 


ams cwa_calib_corr_tp = 


{ 








/* 


make a binary Tiodel 




0. 






/* 


no leniency 




0, 






/* 


don't measure angle 




5. 






/* 


default bias 




1. 






/♦ 


left tail at 1% 




1, 








right tail at 1% 




50 






/* 


threshold, not used 




4 , 


4, 


/♦ 


model 


resolution 




0, 






/• 






0, 






/• 


no angle training 




0. 






/• 


ditto 




0. 


0, 


/* 


no St 


andard deviation thr 




12 


8, 


/• 


care 


threshold for mask t 






0 




/♦ 


NC flags set 



".g ror statistics 



/* (do not bother to initialized reser-^-ed fields; 



static cse_params 

cwa_calib_sp { cse_absolute. cse_abs_binar;/, NO. NC, 1, 150, lOOO) 

/♦ caliper used to find more accurate position estimates of edge 
♦ positions 
*/ 

static 

cclp_constraint cwa_f ind_target_gcs [1] = 

{ 

{CCL?_CONTRA£T, 0, 1000, 0, 100. 0 *• , 

\ ; 



94 



BNSDOCID: <WO 98181 1 7A2_L> 



SUBSTITUTE SHEET (RULE 26) 



wo 98/18117 



PCTAJS97/18268 



ccip_caliper clp_f ind_carget_inicial_search = ( 
LCOSE_CALIPER_WIDTH. /* Search Length •/ 
LOOSE_CAi-IPER_PROJi:CTION_LENGTH. /• Projection Lenc: 



2. 
0, 

10., /* 
EDGE_THRESHOLD, 
CCI*?_DONT_CARE , 
CCLP_NO_PAIRS , 
CCL?_DEFAULT, 
8, 

TRUE, /* 
TRUE, /♦ 
1, 



/♦ Edge Fiicer Size 

/* Edge r ilcer Leniency 
Expected Size •/ 

/♦ Contrast threshold 

/* El Polarity 

/♦ E2 Polarity 

/♦ Optimization 

/* Bits per pixel */ 
Window Rotation ♦/ 
Interpolate •/ 

/* Number of contramts 



cwa_f ind_target_gcs , /♦ Constraints 
NULL, /♦ Pixel Map 

0, 0, 0 



static */ 

cip_f ind_target_f ine_se 
riGHT_CALI?ER_Wir;TH. /* Search -e 
riGHT CALIPER PRCJECriCM LEN'G' 



ccip_calipe 



ar: 
nc: 



2, 
0, 

10.. /* 
EDGE_THRESHOLD. 
CCL?_DONT_CARE. 
CCLP_NO_PAIRS , 
CCL?_DEFAL'LT. 
8, 

TRUE, /• 
TRUE, /* 



/• Edge Fi- 
/* Edge ?i: 
Exoected Size 



?r: 
Si: 
Ler 



/♦ Contrast 
/* El Polar i 
/♦ E2 Pclari 
/♦ Optimzat 
/♦ Bits per 
Window Rctaticn 
Interpolate 
1. /♦ Number of 

cwa_f ind_target_gcs , /* Constrain 
NULL, /* Pixel Map 

0, 0, 0 



resncic 



/♦ static ♦/ 

cclp_caliper clp_f ind_target_compute_subpixel = { 
TIGHT_CALIPER_WIDTH. /• Search Length ♦/ 
!,/♦ Projection Length */ 
2, /♦ Edge Filter Size 

0 , /* Edge Filter Leniency ♦ . 

10., /♦ Expected Size 

EDGE_THRESHOLD . /* Contrast: threshold 

CCL?_DONT_CARE, /♦ El Polarity 

CCLP_NO_PAIRS, /* E2 Polarity 

CCL?_DEFAULT, /* Optimization 

8, /• Bits per pixel *; 

FALSE, /* Window Rotation •/ 
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FALSE. /♦ Incerpclace 

^' /* Number of concramco * 

cwa_f ind_cargec_gcs, /* Constraincs 
NULL, /* Pixel Map 

0, 0, 0 



/* make an image of a rocaced center of a crosshair, chis in;age is 

* used to train a model to search for the crosshair in ar/i^iage* 

* img - cip_buffer where rotated crosshair will be stored 

* orientation^in^tenths - orientation of rotated crosshair m tenths 

of degrees 

* returns img 
*/ 

cip_buf fer *make_cross_hair_center_image 
(cip_buffer ♦img, int orientation_in_ter.ths ) 

cip_buffer win, ♦xhair =NULL; 
cct_signal sig; 

NO_REGISTHH i xhair: ; 

if {limg) 

cct__error (CGEN^ERR^aATARG; ; 

if {sig=cct_catch;0) ) 
goto done ; 



which will contain a 



/* allocate a temporary cip buffer {:chair 

♦ non-rotated crosshairs, and then use this temporar 

♦ as a source for cir;_rctate ^^i^^^^^i 
*/ 

= ^^?_create (itr.c- >width*l , iT.c- ^heicht*! 3 ■ 
cip^set (xhair. 0} ; 

cip^window (xhair, iwin, 0, 0 , xhair- >width/2 . xhair- >heighz. 2: 
cip_3et (&win, 255) ; 

cip_window{ xhair. &win, xhair- >width/2 . xhair- > height/ 2 , 

xhair - >width-^2 , xhair - >height/2 } ; 
cip_set{&win.255) ; 

cip_rotate_centered(xhair , img, orientaticn_in_t2nthsj ; 
done : 

if (xhair) cip_delete (xhair } , xhair^NL'LL; 
if (sig) cct_throw(sig) ; 

return img; 



/* construct a synthetic image and use search tc find a zrzss est— ate 

* of the center position of the crosshair center 

* we expect that the synthetic model will be accurate zz 
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♦ apprcximaceiy 1 pixel, and therefore, ::his sear-r. rc'--ir.e sr.culz 

* localize che cencer co approximacely 1 pixel 

• img - image ccncaining a rocaced crosshair 

* oriencaLion_in_cenchs - oriencacicn of rocaced crosshair ir. cer.chs 

• of degrees 

♦ res - pomcer to cwa_point where result will be stored 

♦ gross_locate_crosshair_cencer_in_image i) returns ir.t signifying 

♦ whether or not it found a crosshair 
*/ 

static int gross_locate_crosshair_center_in_image 
(cip_buffer ♦img. int orientacion_in_tenths . cva _pcint ♦res) 

{ 

cip buffer • targets NULL; 
cse_model *mdl=NULL; 
cct_signal sig; 
cse_results cse_res; 
int ans ; 

NO_RHGIS"?. -.target) ; 
NO_R£GXSr£R irr.dl, ; 

if (sig = cct_cattr. :o,' ; 
goto done; 

target = cip_create (GRCSS_TAHGET_S:2Z. 3?.CSS_T.\?.GET_SIZE. 3" ; 

make_crcss_hair_center__imace (target, crientation_in_tentns : ; 

mdl s= ctr_train_:nodel ' target . Z 0, C, 0. icwa_ca_iz_ctrr_tp 

cse_area_searcr. '. in;g, mdl. icwa_calii::_sp . £tcse_res ' ; 
if (cwa_f ind_target_debug_f lac i C:-:l. 

printf. "\d \d \d Vd\n" , 

cse res .X, cse_res . y. C3e_res . score . tse^res . fc'jr.d; ; 

res->x = 1 . • (GRCSS_TARGE'r_3IZE/2 - CIA__aND15 »'cse_res . x . n; " ; 
res->y = i . • (GRCSS_TARGET_5:2:E/2 • c:a_RNI}16 :cse_res . y .n; ;• ; 
done : 

if (target) cip_deiete ( target ) . target = NULL; 
if (mdl) ctr_delete_mcdel (mdl) . mdl=NULL; 
if (sig) cct_throw (sig) ; 

return cse_res . found; 

} 

/• compute distance from a point to the closest point on a Line -/ 
static double distance_f rom ooint_to_line (cwa_?oint 'pl. cwa_line 'li: 

{ 

cwa_line 12 ; 
cwa^oint p2 ; 

12.x = pl->x; 
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12.7 = pl->y; 
12. C = ll->t + PI/2; 
cwa_lines_co_point (11, &12 . &p2) ; 
recum (sqrt ( (pl->x-p2.x) * (pl->x-p2,x) 
^ (pl->y-p2 .y) * (pl->y-p2 .y) ) } , 

/* Fin a line co an array of points ♦/ 

static double f it_line_to_points 

(cwa_point •pts, int n_points. cwa_line ♦line) 

int j; 

doxible X3um=0.0, ysum=0.0, xcent, ycent; 
double a, b, c, theta, xp, yp; 
cwa_point ♦ptr; 

if (njpoints < 2) 

cct_error {O?A_ERK_N0_INTERSECT) ; 

for {j=o, per = pts; 3<n_pcLP.zs: ?rr— 
if {cwa_f ir.d_target_debug_f lag i 0x1; 

pr-ntf ("Vf Vf \n'\pt:r->x.ptr->y: . 
xsum ptr->x; 
ysum += ptr->v; 

} 

xcent = xsum / appoints; 
ycent = ysum / n_points; 

for (j=0, ptr = pts, a = b=c = 0.; := <r._j;cir.t3 ; : — , ccr— - 
xp = ptr->x - xcent; 
VP = ptr->y - ycant ; 
a += xp ♦ xp; 
b 2 * xp * yp; 

C ^= yp * yp; 

} 

theta = atan2(b. a-c) / 2.0; 

line->x = xcent; 
line->y = ycent; 
line->t = theta; 

if (cwa_find_target_debug_flag & 0x1) { 

printf ( '*f it_line_to_j)oints returned: Vf'\ 
(a*cos (theta+PI/2) *cos ( theta+PZ/2 ) + 
b*cos {tfaeta+PI/2) ♦sin {theta-t-PI/2) - 
c*sin(theta+PI/2) ♦sin { theta*Pi/2) • /n ^points! ; 
cd_showas ( ( int) line , "cwa line" ) ; 

}. 

return ( (a*cos ( theta-K?I/2 ) ♦cos {theta-^PI/z: - 
b*cos (theta+PI/2) ♦sin (theta+PI/2 ) - 
c*sin( theta+PI/2) ^sm ; theta+?I/2 ) ; /n_points; ; 
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static int print_list_of joints 
(cwa_poi::t •val. int num_vals) 

{ 

int i ; 

for (i = 0; i < num_vals; i-^*) 

printf {-\f VfXn-. val (i; ,x, vai li) .y) ; 

} 

/♦ calib_orient_f rom^angle () returns CALIB^VERT or CALZ3_HORIZ 

* depending upon the orientation (measured in degrees) zz a line. Its 

* purpose is to determine whether we need to use a vertical or 

* horizontal quadratic subpixel edge interpolation 
*/ 

static calib_orient calib_orient_frcm_angle {double angle in_degreesi 

{ 

double angle_in_rad; 

angle_in_rad = angle_in_degrees ♦ ?I/ 180 . 

if (f abs (cos iangle_in_rad; . > 

fabs (sin (angle_in_rad; . 

return CALI3_VERT; 
retum CALI3_HCR:Z; 

} 

/* ccmputss subpixel edge pcsiticr.s given an image anc a r.e-ghbarhcod 

* (ap_x, ap_y) to search for an edge. Depending upcn the 

* orientaticn_in_degrees , ccr.pute_subpixel_pcsiticn '! I will -sither 

* perform a vertical or horizontal subpixel edge estirvaticn . Subpixel 

* edge positions are computed ir. the same manner as the hcjndary 

* tracher (using 7 neighbcrir.g pixels, and 7 differences where 

* we quadratically interpolate the edge pcsiticn from at least three 

* differences. 

* We compute a seven first differences Tust tc be safe. There is no 

* guarantee that the computed caliper position correspcr.ds to the maximum 

* 1st difference {because we're using calipers wich filter size 2; 

* img - cip_buffer containing image of crosshair target 

* ap_x,ap_y - application pcint where we sample image grey values 

* actually, we sample grey values at the application 

* point and -<-/- 2 from the application point (maybe we 

* should sample at 4, who knows 

* orientation_in_degrees - orientation of caliper which has been applied 

* to find this application point. Since we 

* only want to call subpixel interpolation 

* at the right place, we suggest using 

* on-axis calipers 

* pos_x, pos_y - pointers to doubles where 

* compute_subpixel_position { ) will store result 
*/ 

int compute_subpixel_^osition 
(cip^buffer *img, int ap_x, int ap_y, 

double orientation_in_degrees , double ♦pos_x. double •wOS_y') 
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inc P0,pi.p2,p3.p4.p5,p6,p7; 

d0.dl,d2,d3,d4,d5,d6, d_max, d_n^ir.; 
caiib_orienL direction; 

if ( cwa_f ind_carget_debug_f lag i 0x1 ;■ 

prmrf ( "compuce_subpixei_pos*cicr. called wi-r. Vd V;c 
ap_x, ap_y, oriencacicn_in_degrees . pcs_x, CCS v . 

if (ap_x < 4 I I 

ap_x > img->width-4 || 
ap_y < 4 I I 

^P_y > img- >height-4) ( 
return 0; 



/* should we sample honccntally cr vertiral 
direction = calib_orient_f rem angle -crienta: 



f (direction == CALI3_nCRrz: ( 
pO = * (img- >rat [ap_y-4 ] -img- >x_cf f cet-ac >: 
pl:=* (img->rat iap_y-3; -ing- >x_cf f set -ac~;c 
p2 = * (img- >rat [ap_y-2 ■ -img- >x_cf f set -at~:-;. 
p3=^* (img- >rat [ap_y- 1] *imc- >x_of f set -at";-:. 
p4 = * (img->rat [ap_y] *img- >:<_cf f set *at ;<~: 
p5=* (img->rat [ap_y*i; -img- >x_cffset-ac x. 
?6 = * {img->rac Lap_y-2] -^img- >x_af f set -apjc; 
p7 = * {img->rac Lap_y^3i *img- >x_of f set-^ap'xi 

dO=pi-pC 
dl=p2-pl 
d2=p3-p: 
d3=p4-p3 
d4=p5-p4 
d5=p6-p5 
d6=p7-p6 



d_max = max(dO,max(dl,maxtd2.maxid3,maxid4.max;d5.d6; 
d_min = niin (dO , mm (dl, mm (d2 , min ';d3 . mm id4 . min (d5 . d6 ; 

if (-d_min > d^max) { 
dO=-dO; 
dl=-dl 
d2=-d2 
d3=-d3 
d4=-d4 
d5=-d5 
d6»-d6 

d_max =-d_miii; 

} 



♦pos_x = ap_x*l. -0.5; 

/* If we * re at a maximum return the suboixel cosiiicn 
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:.f (d_(na:< > £DGE_TKRESHOLD) ( 

if (dl == d_max d3 := d_maxJ ( 

♦pos_y = ap_y cz^parf ic JdC . cll . J "o 553o.-Z.. 
return 1; 

} else if {d2 == d_max ii d*l : = d__max; ( 

•pos_y = ap_y + C2_parf it {dl , dZ , d3 ■ /5 =5 35 1 . ; 
return 1 ; 

} else if (d3 d_max &tt d5 = d_max) { 

*POS_y = ap_y + C2_parf ii (d2 , d3 . d4 ; / 6 5 =36 . ; 
return l ; 

} else if (d4 d_max && d6 := d_max) { 

*pos_y = ap_y + C2_parf it (d3 , d4 . dS) /65536 . --I . ; 

return 1; 
} else if (d5 == d_niax) ( 

••pos^y = ap_y + C3_parf it {d4 . d5 , d6} /65536 . -r2 . ; 

return 1; 

} 

/♦ If we're not az a maxiT.ur., recurr. : scmetr.ir.c 3 vrzr.= 
*/ 

return 0; 



/ else { 



?0 = 




{img- 


>raz :ap_y: 


-:.nic - 


>x_3ff sec 


-ac_x 


pl= 


* 


(img- 


>rat [ap_y] 


^img - 


>x_cf f sec 


-ap_x- ; 


p2 = 




( img- 


>r3t iap_yl 


-in-.g- 


>x_of f sec 


•ap_x - - 


p3 = 




{ img- 


>rat iap_y 1 


•inig- 


>x_of f set 


-ap_x- 1 


p4 = 


* 


(img- 


>rat [ap_y] 


■^ing- 


>x_cff set 


-ap_x: ; 


p5 = 


* 


i img- 


>rat [ap_yi 


-irrg- 


:>x_of f set 


■.-ap_x-l 


p6 = 




vimg- 


>rat [ap_y] 


*inag- 


>x_of f set 


*ap_x-Z 


p7 = 


* 


( itng- 


>rat Cap_y] 


*img- 


>x_af f set 


*ap_x-3 



dO=pl -pO 
dl=p2-pl 
d2=p3-p2 
d3=p4-p3 
d4=pS -p4 
d5=sp6-p5 
d6=p7-p6 



d_max =s max ( dC , max ( dl , max ( d2 , max i d3 . max i d4 . max { dS . d6 ))));) ; 
d min = min (dO , min (dl , mir* (dZ . min (d3 , min (d4 , min i dS , d6 • M ) ; ; ; 



if {-d_min > d_max) ( 
dO=-dO; 
dl=-dl 
d2=-d2 
d3=-d3 
d4=-d4 
d5=-d5 
d6=-d6 

d max =-d min; 



/• If we're at a maximum, compute the suhpixel posit:.cn and 
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* recum 1 
•/ . 

♦pos_y = ap__y*i. ^ 0,5,- 

if (d_max > SDGE_THRESHOLD) { 

if (dl d_max &i d3 := d_max) ( 

•pos_x = ap_x*l. + cr_parfiC{d0,dl.d2;/65536. -2. ; 
re cum l; 

} 

else if (d2 d_max d4 != d_max) { 

*pos_x = ap_x*l. cz_parfit (dl,d2,d3) /65536. -1. ; 
return l; , 

} 

else if (d3 == d_max dS •= d_max: { 

*pos_x = ap_x*l. 4. c= jparfir (d2.d3."d4}/65536. ; 
re cum 1; 

} 

else if (d4 == d_niax &.u dS := d_aiax} < 

*pos_x = ap_x*l. * ==_?arf It .'d: , d-;. . 65 = 3o . 
re cum 1 

! 

else if (dS == d_max: 

•pos_x = ap_x*:. - cc_parf It -.G4. d5. do . 65535 . . ; 
recum. i; 

} 

/• If we're noc az a maxiT.ur., scn^ezhir.c: ' s a--^ 3 

recum 0; 

I 

} 



, * enumerate^Oincs^ar.d_fic_iine_alcn9^ray 0 er.umeraces edge pcmcs 

* along a ray from a scare _pcmc co an end_pcinc by usmg calipers 

* (and possibly subpixel ir.cerpolation) co compuce'edge posicions 

* enumeracej)oinC3_and_fic__lme_along_ray() concinues enumeracmg 

* poincs uncil one of che following condicions occurs: 

* 0 . we reach che end_j3oinc 

* 1. no edge is found by che caliper 

* 2. more Chan one edge is found by che caliper and che highesc 

scoring edge is less Chan cwice che score of che second highesc 
scoring edge (chis 2X comes from che CONFUSION_THRESHOLD} 

* 3. che computed edge poinc is an ouclier (che discance becween che 

edge poinc and che nominal line is larger chan threshold) 

* 4. we run off che cip_buffer (screen) 

* Afcer ic has enumeraced che edge pcincs, it chen calls 

* f ic_poincs_co_line Co compuce che cpcimal lir.e escimace 

* img - cip_buffer concaining a crosshair cargec 
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scarrjcinc. enci_poinc - define region alcna whirh zz apply zalipers 
nominal_line, threshold - used cc decermine whether an ecce pc^nr 

is an outlier land ccnseq\:enziy end zhe 
enumeracicn of edge points) . The cutlier 
test involves checking whether the 
distance from the point to the 
step size - frequency of edge point sampling. Since we want zz 

evenly sample the lines, we move step_si=e nuroer cf 
pels in the larger direction (and a fractional number 
of pels in the other direction) . This approach provides 
even sampling assuming we are doing on -axes calipering 
or parabolic subpixel interpolation 
3earch_caliper - caliper used to search for edge points 
compuced_line - cwa_line where result will be returned 
enumerated_points - optional array of cwa _jioints where ftund edge 

points would be stored 
num_points - pointer to an int where the number of zz'^.d pcmts 
would be stored 

do_subpixel_interp - flag characterizing whether we wanr zz perfzrrr. 

suboixel inzerpclaz.zn after azz./izr zr.e zaliz 



static int enumerate__pcints_anc_f iz_line_a-zng_rav* 
(cip_buffer -img. cwa^pcmt *starz _ooint, cwa_pcinz •enz_zzLni 
cwa_line ♦nominal_line , double zhreshcld. double szep_5ize. 
cclp_caliper •search_caliper , cwa_line *compuzed_line , zwa_zcinz 
♦enumerated oomts, int ♦num_point3. int dc_subpixel_interp 

{ 

int i.j,k; 

double maxAbsCosSm; 

cwa_point ap_point. •enuT_zcinz. vec; 
int ap_x. ap_y; 

cclo results res_h IM_EDGES * Z -1 ! 

cclp_params cpp = { EDGE_TH?.E3HCL2 * 1 3 , N'_EE3E3, C ;: 

doxible cclp_angle; 

int num_s amp 1 e s . t mp 

double caliper_vert ; 

cct_signal sig; 

cwa_line sampling_line ; 

if (cwa_f ind_target_debug_f lag & 0x1) ( 

printf (•*enumerated_points_and_f it_line_alcng_ray called witn .n" 

printf(**img Vx start_point Vf Vf end_point ^f tf\n*\ 

img , start_point - >x . s tart^point - >y . end _point - >x , end_?cint - >y ) 
printf ("nominal_line Vf Vf t Vf threshold Vf\n'\ 

nominal_line->x, ncminal_line- >y . nominal_line- >z, threshold; ; 

} 

/♦ Compute the sampling line ♦/ 
sampling_line.x - start _point-:>x; 
sampling_line. y = start_point - >y ; 

sampling_line. t = atan2 (end_pcint - >y-start_point- >y . 

end_point - >x - start_j;cint - >x; ; 
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/'* Compuce the seep wh^ch is i.-:tegral ir. x — — ^----^-^3 
*/ " ' 

if (fabs (cos (aampiing^lme. ci ; > tabs i : samplir.g_l me . c . 

.'naxAbsCosSi.-i = fabs i cos ( sampling line.t;'; 
else 

maxAbsCosSin = fcibs (sin (sampling Ime-C)) ; 

vec.x = (cos{sarapling_iine-t:) /maxAbsCosSin ; * srep size; 
vec.y =. (sm (sampling^line. c) /maxAbsCcsSin) * scep^size; 

/* Compute the number of samples which we will be mak' nq 
*/ 

num_samples = 

max (oibs ( (int) (scart:_point- >x - end_pcint ->;<): . 

abs ( (mc) (scarc^point - >y - end coinc->v* 
step_size; 

if { ! enumeraned_pcints; 
enumerated_pcint3 = 

(cwa^pcmr *) chp_alloca ,'nurr._sarT'.p:es*sizecf ■ cwa ccmr 

if ( I num_points ; 
aum_points = itmp; 

cclp_angle = sampling_line . 1*130 . .'?:.9C . ; 

if (do_subpixel_int:erpi { 

if (calib_orier:t_f rDm_an9le ccip_ar.cle: ~ ~ C.=il;3 

cclp^angie = 0; . ~ 
else cclp angle = 90; 

} 

if (cwa_f ind_targec_debug_f lac i Oxl; 
printf ( "cclp_angle Vf \r." , c=lp_angle; ; 



■3" 



/♦ Inner Loop: Enumerate edge points 
*/ 

for (i = 0, ap_point,x = snart_poini: - >x, ap^pcmc.y = 
start_point->y, enum_point = enumerated^pcints , 
♦num_points = 0 ; 
i < num_samples ; 

i++, ap_point.x vec.x, ap_point.y 'r^c.y. enum^pcir.t , 
(♦num_points) { 

ap_x = (int) (ap_point.x * 0.5) ; 
^P_y = (int) {ap_point.y * o.S); 

if {cwa_f ind_ targe c_debug_ flag i Oxli 

printf [ "ap_x td ap_y %d\n" , ap_x, ac y* ; 
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res h[0]-fcund = res hO.fsur.d = 0; 



/* cip_cransfonn throws CGEN_ERR_3ADARG, Cr?_ERP-_?ElA23R . 

* CCLP_ERR_rIT when caliper region excer.ds pasr che "ccunds zz 

* the cip_buffer. We want to break off the search at this 

• point anyways, so we catch the error and ]urrp tc iittir.c a 

♦ line to the points 
•/ 

if (sig = cct_catch{CGEN_ERR_3ADARG) ; 
goto fit_line; 

if (sig = cct_catch(CIP_ERR_PELADDRl ) 
goto fit_line; 

if (sig = cct_catch (CCL?_ERR_FIT; : 
goto fit_line; 

cclp^apply 

( search_cal iper . inc.ap_>:. ap_y. zclc^angl^, i-cc. res r, 

cct_end{CCl?_ERR_~IT: ; 
cct_end(CI?_ERil_?ELAECR; ; 
CCt3end'.CGEN_ERR_3ADARG:- ; 

/* check if edge is not four.c 
if ( !res_h[ol .found) 
goto fit_line; 

/• check if more than cr.e edge is fcur.c CCM?USi::S' * 
if (res_h[li .four.d Scu 

res_hC0l .score < CCNrU3IC?:_THRESHCLD*res_h !ll .score; { 
if (cwa_f ind_ tar get _debuc_ flag & Oxl. { 
cd_showas i res_h, "cclp_results" ; ; 
cd_showas t »res_h i 1 ] , "cclo_result5 " . . 

} 

goto fit_line; 

} 

if (cwa_f ind_target_debug_f lag & 0x1} { 

printf (-ap_x %d ap_y \d angle %f pes Vf\n'\ 

ap_x, ap_y. cclp_angle , res_h [ 0] .position) ; 
if (fabs (res_h[0] .position) > l.){ 
cd_showas (res_h, "cclp_resuits" 1 ; 
cd_showas ( tres_h(l] , "cclp_results" ; ; 

) 

} 

/♦ compute the subpixel point measured by the caliper • " 

enxim_point - >x = 

ap_x + O.S + res_h [0] . position-cos icclp_angle*?I/ 190J ; 
enum_po int - >y = 
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ap_y * G,5 - res h [ 0] . pc3iricr.*sir. rrlc a-r'^*"- ■ 
if (do_3ubpxxei_interp) ( 



if ( ' -ornpute_3ubpixel_positicr. 

(img, (mc) enum_poinL - >x , iinc; enum_poinr - >v. 
cclp_angle, &enum_poir*r - >x, ienum_po*r.c - >v* 
goto fic_line; 
If (cwa_f ind_targec_debug_f lag St 0x1) 

prmcf { "compuce subpixei posicion returned tf .n", 
^ enum _point- >x, enum_poinc- >y) ; 

#endif 

/♦ check if the point is wichm the ACCEPTABLE REGICM 
* (wich:in distance threshold of the nominal line 



If iciistance_frcm^cint_tc_lineer.ur._pcir.r. noxinal^lme: > 
threshold) 
goto fit^lme; 



/• fit a line to the points and return : if we enrc'-L-tered 
• any bad points, return 1 rtnerwise 



:_line_to_pcints 

(enumerated_points, *nur,_pcint3 . rcmputed line; 



static mt round nuru:er Idouhl-^ x. 
{ 

If (x > 0) 

return (int) (xi-0.5) ; 
else 

return imt J ix-0 .5) ; 

} 



* average two orientations but take into account that orientations 

* mod PI are equivalent 
♦/ 

tatic double average_two_orientat ions -double oriental, double orient Si- 
double diff,res; 
int modDiff; 



if {cwa_find_target_debug_flag & 0x1) 
printf("avg %f Vf orient_l , orient_2 ) ; 

diff = orient_2 - orient_l; 

modDiff s round number (diff/ PI ; 

res= ; (orient_l-Qrient_2 -modl^if f *?: , ;2) ; 
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if cwa_f ind__iar'gec_debu9_f I39 i 

princfC = \f\n",resi; 
re cum res ; 



/♦ coT.puce angle in ai^emacive coordinates giver, by zhe -ap 
*/ 

double ::ransf orm_angle_according_to_calibracion_T.ap 
(priv_-ransf orm 'map. double iT.g_oriencacicn • 

{ 

double realSin, realCos ; 

C2_transf ormPoinc (map, cos i irr.g_crientat:icn) . sir. ( imc_crier.-acicrJ . 

&realCos , irealSin) ; 
recurn(acan2 (realSin, reaiCos) ) ; 

} 

/♦ average che orientiacions of zhe four rays zc ccmpuce zr.e obsz 

♦ coaiposite orientation esziT.ate. Given the orienraticns zz the feu: 

• image rays (the rays should be orieniec in T.ulziples z : ?Z. 2 cut 

• witb eacb other:, compute the "average* crientatitn. "irst. the 

* orientations are aligned vith iT.g_cr lent ! C ! sc if i-g_"r lenr ! : * 

♦ characterized a horitcntal line, then all zz the angles T.ay be 

* shifted by ?I. - to become horizontal, tnen we average ihe.resulti: 

* orientations - 

• This function transforms the ir.ace 

♦ orientations into the physical domain m order to explcit the 

• constraint that the physical lines are orthogonal 
*/ 

static double average^f our_ir.ace_orientations 
(cwa_calib ♦calib, double • :.mg orient: 

{ 

double ohys orient l-i ', , avg_chys_crient . avg_img_orient . 

♦ First, transform angles from image ccordiantes to physical 

♦ coordinates 
*/ 

for (i = 0; i < 4; i**i 
phys_orient [i] = 

trans f orm_ang ie_ac cording_t o_cal ibrac ion_map 
{ ( (priv_cwa_calib * = calib; ->map. img_orient [ i ] ! ; 

if {cwa_f ind_target_debuc_f lac Sl 0x1) 
for (i 1; 1 < 4; i*- 

printf fVf \n" ,phys_crient [i*. ^ ; 

/♦ Shift orthogonal angles by in order to line up with the 

♦ first orientation 
*/ 

for ii = 1; i < 4; i-^.; 

if ( fabs ism lphys_or lent ; i*. - phys_orient ;0l >scrt .:.3; . 
phys_orient (i 1 ?1,'2; 
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/• average the orientations • 
avg_phys_orient = 

average^cwo^oriencations 

{ average_two_oriencaticns .phys crier.c [0] . pnvs ' 

average_two_oriencacions -phys^Drient [2] - phvs~3rienc : 3 * ■ • 

/* compute the orientation m image cccrdmates corresccnding to 
* the averaged orientation 
*/ 

avg_img_orienc = 

transform_angle_acccrding_to_caiibration. map 

( ( (priv_cwa_calib calib) - >inv_map, avg_^hys_orient^ ; 

if (cwa_find_target_debug_f lag & Oxi) 

printffphys %f img * f \n" , avg_phys_orient , avg_^mg_orient ) .- 

return avg_img_orient ; 



/♦ compute the image orientation w--- --p ^- -^.^ - 

* is orthogonal to another imace ori'*':"at 

♦ / 

static double i!nage_ncrTTial tc_image ancrle 
(cwa_calib •calib, dcubi« -mc c-*^-'-" 

return 

{transform_angle_ac=trding_-tc_calibration_map 
( ;priv__cwa_calib 'ioalib - >inv niao. 

{transf = rm_angle_acc=rding_t3_ca:lbraticn_maD 
( Mpriv_cwa_talib * : talib . - >map . ima trient;'* ?:,:)■'• 



compute_center_oosition,and_angle_assuming_four_guaranteed_good ravs ■ - 

* improve estimate of the crosshair position and orientation bv 

* searching four regions of guaranteed good length (50 pixels)' 

* eminacing from the crosshair center. Lines are fit to the edge 

* points along the four regions, and then the orientations of these 

* lines are "averaged". T^o orthogonal lines are computed bv 

* averaging pairs of the four rays Finally, the position estimate is 

* computed by intersecting the two orthogonal lines 

* img - cip_buffer containing an image of a trosshair target 

* calib - cwa_calib object characterizing pixel non- rectangular itv 

* estimated_ctr_img - estimated position (given in image 

coordinates) of crosshair target (computed in steo 1) 

* estimated^orientacionjhys - user- supplied orientation estimate 

(given m physical coordinates) 

* guaranteed__good_distance - length (m pels) of guarateed good 

region extending outward from the or-g-- 

* computed_center - structure where computed position will be stored 
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compuLed_orier.cacicn_phys - double where ccmpuied orier.za-ic: 

physical coordinaces.* will be scored 
good_rays - array of four cwa_iine scruccures where good rays 

(lines fie zz sampled edge pcmcs m che gccd regicr.. 
will be scored 

/ 

scacic int compuce_cencer_posicicn_and_angle_assuming_f oi:r_cuarar.ceed_gcod_rays 
(cip buffer 'img, cwa_calib ♦caiib. cwa_poinc •escimated_ccr_img . double 

escimaced oriencacion_phys , double num_guaranceed^good_poinc5 . 

cwa_point ♦compuced_cencer , double ♦compuced_oriencation_phys , 

cwa_line *good_rays) 

{ 

int i ; 

dotoble angle [4 ); 

cwa_point starc_point [-4 ] . end_poinc (4 ] ; 
cwa_line compuced_Line [4 ] . acminal_line ; 
double img_angie[4l , angle_horiz_iine ; 

angle [0] = esc Lmaced_oriencacior._pr.ys ; 

angle[l] = escimaced_oriencac-cr._phyS'?: . 1 ; 

angle [2] = escimaced_crier.cacicr._phys-?: . 

angle [31 = esciniaced_crier.cacicr._phys-?: * l . 5 ; 

/* We define che 5C pel GUAJLP^TEHZ 3CCC REGION ir. cerr.s ::: a 

♦ VALID REGION so chac we car. use 

♦ Che enumerate_poincs_and__f ic_line_along_ray T: f-or.ccicr. 

♦ If we're sampling along a l:.r.e and we wane cc seep sanpling 

♦ after 50 pels, Chen we can use a nominal line which is ncrrr-al 

♦ CO and passes chrcuch rhe scare peine, and a chreshclc zz 5C 
•/ 

nQminal_line.x = escir.aced_ccr_i:ng- >x ; 
nominal line.v = escinnaced ccr img->y; 



/♦ The passed criencacicn is giver 



m onvsica. cccriinaces 



for (i = 0; i < 4; 

angle [i] = cransf orni_angle_acccrdinc_co_calibracion_n\ap 
( ( (priv_cwa_calib »>calib; - >inv_map. angle [i; ) ; 

for (i = 0; i < 4 ; 1*-^) ( 

/♦ The sampling region begins ac che pome a small discanc* 

• away from che crosshair center 

♦ (CIRaJLAR_BUrFER_ZCNE_AROUND_GENTER; CO be exacc, and 

* extends out co guaranceed_good_discance from che center 
•/ 

scart^point (i] .X = estimated_ctr_img- >x -^ 

CIRCUIAR_BXJFrER_ZONE_AROUND_CENTER*cos (angle [ij !• ; 

start_point [i] - y = estimaced_ccr_img- >y * 

CIRaJIJUi_BUrFER_ZCNE_AROUND_CElTrER*sin (angle [il } ; 

endjoint [i] .x = est-mated_ccr_img- >x 

num_guaranteed_gcod_points*cos (angle (i] ; 

end^point [i] .y = estimaced_ccr_img- >y * 
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num_guarant:eed_gccd_poiiit:3 *sin (angle ; ' 
nominal _1 ine. c = angle {i] t-PI/Z ; 

enumerace_poincs_and_f it_line_along_ray 

( img, &scarr_poinc til , &end_c2inc [i ; . incnmal line. 
num_guaranteed_good_poincs . 

STEP_SIZE_USED_T0_ENUMERATE_?OINTS_IN_3COD_RZ3:cr:*2 . . 
&clp_f ind_targec_initial_search, 
tcompuced^line [i] , NULL, NULL. 0) ; 
img_angle [i] = computed_line Ci ] . t ; 

cu_copy {&computed_iine Ci] , &gaod_rays [i] ,sizeof 'cwa ■ ^-e"' ) * 
} ■ ' ' 

/♦ average the angles */ 

angle_horiz_liae = 

average_four_image_oriencat:ions ;callb, inc ancle. ; 

ir ( cwa_f ind_targec_decug_f lag i 3x1; 
prmcf ( "average Vf Vf Vf Vr'n". 

img_angle [0] . i:ng_angle ;i ; . -r.c_angle [Z] , it.c: an-le ; 
angle_horiz_line) ; 

average che fir rays 

comput:ed_iine lO! 
computed_line [0] 
ccmpuced_line [1] 
ccmpuced^line [1] 
ccmpuT:ed_llne [0] 
ccmpuced_line [ll 

* compute the intersected point *, 

cwa_lines_co_pcint { StComputed_line [C] , iccir.puted line [1] , computed center: ; 

/♦ fix the orientations of the good rays to corresocnc to the 

* "average" orientation 
*/ 

good_rays [01 . t = good_rays [2 ] , t = angle_horiz_line ; 

*computed_orientation_phys = 

trans fonn_angie_according_to_calibration_map 

( ( (priv_cwa_caiib •) caiib) - >map , angle_hori2_iine) ; 
good_rays(l) .t = good_rays(31 .t = 

transf orni_angie_according_to_calibration_map 

( ( (priv_cwa_calib ♦ ) caiib) - >inv_map, 
•compuned_orientation Dhvs-»-?I/2) ; 

} 

/* Compute the two intersections between a circle and a line if the 

* circle and line intersect. Xf the circle and line do not 

♦ intersection, signal CMA_ERR_NC_ INTERSECT I ON 
V 



.X = (ccmputed_line [Oi . X - 
.y = {computed_line [0] . y ^ 
-x = (computed_line [1 ] . X -r 
.y = (computed_line [1] . y * 
.C = angle_hcrit_lin2 ; 
-t = image_ncrTT:al_tc_imace 



computed line [Z] 2 : 
computed_line [2] . y' :2 ; 
computed_line [l] . x; ■ 2 : 
computed_line [ 3 ] . y ; ' 2 ; 

ancle calib , an^le hone 
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szazLC inc incsrsect_iine_ar.d_cirrle : cwa_li.-ie •line. cva_j:rir.i *ct. 

double radius, cwa_poir.c •res_l 

{ 

double angle_co_line . discance_r 3_lir.e , angie_ro_cci.-r_rr._lir.e . 

angie_to_line = line- >C'»-PI/2 ; 
if ( (co3 <angle__co_line) • (line- >x-pc- >x) * 
sin {angle_co_line )* (line- >y-pc - >y) ) < o; 
angle_to_line ■^= PI; 
distance_co_line = distance^f roni_poinc_co_line (pc , Imei ; 

if (distance_ro_line > (radius - EPSILON_THRESHOLD) • 
ccc_error (OrA_ERR_M0_rNTERSECT) ; 

angle_to_poinc_on_line = acos (dist:ance_zc_line/radii:s: . 

res_l->x = pc->x + 

radius*cos (angie_co_line-t-ancie_rc_pcinc_or._line ) ; 
res_l->y = pc->y + 

radius* sir. ;angle_^c_ line -rang le_-c_ocir.i_cn_l me = 
res_2->x = pc->x 

radius^ccs ;angle__co_line-angle_zc_pcinr_cn_line; .- 
res_2->y = ?t->y * 

radius*sin iangle zz line-angle_tc_pcin-_3n_lir.e. ; 

} 



/* compune_cenrer_posi w-cn_anc_ar.gle_along_guaranceed_gccd_r3ys . ! 

* generace final escimate of crosshair position and orienraricr. by 

* enumerating pcincs along four rays I using four good rays as 

* thresholding lines) . We use tighter calipers because ve trust the 

* positions of the good rays. 

* img - cip_buffer containing an image of a crosshair target 

* calib - cwa_calib object characterizing pixel non- rectangular! ty 

* estimated_ctr_img - estimated position (given in image 

* coordinates) of crosshair target (ccr:puted in 

* seep 2) 

* good_rays - array of four cwa_line structures where gocd rays 

* (lines fit to sampled edge points in the gocd region) 

* will be scored 

* tight threshold - threshold used for determining outlier points 

* with respect to the good rays 

* computed center - structure where computed position will be stored 

* computed orientation_phys - double where computed orientation (in 

* physical coordinates) will be stored 
*/ 

static int corapute_center_position_and_angle_along_guaranteed_good_rays 
(cip_buffer •img, cwa^calib •calib, cwa_point •estimated_rtr_img, 

cwa_line ♦good_rays. double tight_threshold, cwa_j30int 

♦computed_cencer , double •ccmpuced_orientaticn_phys . 

double ♦feature score) 
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int i; 

double angle [4 ] ; 

cwajpoint scarr_j3oini: [4 ] , end_coir.t [4 ] ; 
cwa_line comput:ed_line [4 ] ; 
double img_anglef4] , angle_hori2_lir.e ; 
inc num_enumeraced_points (4 ] ; 

cwa_poinc po3sible_3tart_poincs , enumeracsd^pcmrs 14 ] [MAX mum POINTS] ■ 
double avg_squared_error ; ~* ~ " ' 

if (cwa_f ind_targec_debug_f lag & Oxl) 

princf ("compuce_ceni:er_position__and_angle^alcng_guaran!:eed_good_rays\n") 
for (i = 0; i < 4 ; i-h^) { 

double discance_ro_ceni:er, radius , orien-ation_away__from center; 

/♦ For each ray, we need zo determine what the sanding 

* line should be 

♦ We do this by intersecting the ray with a circle 

* centered at the crosshair center 

♦ Since there are tvc intersection ccmt^ . we need -c 

• determine which one is the real start zz the sanclmc 

* line 

* It cums out that we can sit.cIv tic>: the inters^' 

♦ pome which is closer tc the cent^*- c* "^e ^a" 
*/ " 

discance_to_center = 

distance^fromjpoint^t online ■:estimated_ctr__inig, igcod_rays [i] : ; 

radius = ^qrz iCZRC'JlSJi_^'JT?zil_ZZ^:E_XP.zv>n: 

crRCULAR_3C'r~R_zc:ii:_A;icLT'n:_CE!rrE?-- 

distance_tc_center''distance tc center ; 

intersect_line_and_circle igocd_rays :i: , estinated ctr ixg, 

radius, ipossibie_start_pcint3 • 0] , 

&possible_start_pcint3 [1] ; ; 
if (cwa_poinc_distance 

(tpossible^scart _points [0] , (cwa_point *; igcod_rays [L]) < 
cwa_po in t__di seance 

(&possible_start_point3 [1] , (cwa^pomt • ) igood_rays (ij ) ) 
cu^copy {fi.possible_scart_j3oints [0] , istart_point [i] , 
siteof (cwa_point; ) ; 

else 

cu_copy {&possible_start_points [l] , istart_Doint [i; , 
sizeof (cwa_pcint; ) ; 

/♦ Now, we have to determine which way we should travel along 

* the ray (good_ray[i] .t or good_ray(i] .t+?I 

* Me determine which way by comparing the start point with 

* the center of the good ray 
*/ 

orientation_away_f rom__center = good raysti].t; 
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if ( (cos (oriencac:.on_away_f rom_cencer .* * 

(good_rays [i ] . x-esnimaced_ctr_img - >x. - 
sin (oriencacion_away_f rom_cencer ; • 
(good_rays ii) - y-escimaced_crr_img - >y < C: 
orient:ation_away_f rom_cencer ^= PI; 

end^point [i] -x =» starc^poinc [ i] . x + 

MAX_LZNGTH_DIAGONAL_ACROSS_CI?_BUr?H:H* 

C03 (oriencat:ion_away_f rom_cencer ) ; 
end_poinc [i] .y = starc_poinc f i] . y + 

MAX_LENGTH_D I AGONAL_ACROSS_CI P^BUFFER ♦ 

sin (orient:acion_away_f roin_center ) ; 

if (cwa_f ind_cargec_debug_f lag & 0x1) { 

princf ( '•center tf %f good_ray Vf Vf distance \f .r.". 

escimaced_ccr_img- >x, esLimated_crr_img- >y . 

good_rays[iI . x, gocd_rays f i ■ . y , gccd_rays ! i ! 

dist:ance_co_cencer) ; 
prinzfi-Vf Vf Vf Vfn". 

scarc_r;cinc [ i i . x , scart_cc-r.t ! i ! . y 

ead_j3c ir.t [ i ] . x . end^pc [ i ! . y , 



enunierace_naint3_and_f iz_l ir.e_alcng_ray 

{ img, ist:arr_pc-r*t Li : , iend^pcmt [ i i , icccc_rays ! i ] . 
cight_threshold. 

STE ?_S I ZZ JJS z:D_TO_ enumerate^ AI-L_ ?c ints . 
&clp_f ind_"arget_f ine_search. 
/ ♦&clp_f ir*d_cargec__ccmpuce_sui:pixel . * 
&compuced_lire[i! ,&er.umeraced_pci-Z3:i! '0: . 
tni:m__^enu[neraced_j:oir.ts [L] , 1; 
img_angle[i] = ccmDUted__lir.e : i ] .Z; 

} 

/* For each pair cf opposite rays, use the closest points sc thac 

* we use the same number of points on both sides of the 

* crosshair center 

* Since we don't know why the last edge point became invalid, we 

* skip the neighboring 2 points just to be cn the safe side 
*/ 

num_enumerated_points (0] = num_enumerated_points (2 ! = 

min(num_enumerated_points (Ol , num_enumerated_points [ 2 i : -CUTLIEH_BUrF£R; 
num_enumerated_points [1 ] = num_enumerateG_pcints [3 ] = 

min (num_enumerated_point3 [11 . num_enumerated_points [Z] ■ -CtrrLIER_3UrTER ; 

/* Re -fit lines to these points 

^■^S.squared^error = 0 ; 

for (i = 0 ; i < 4 ; i-^-) { 
double fit error; 
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fit_error => 

f it_line_cojpoints 

(&enumeraced_points I'i] [0] . num_enumeracec_ccinr3 ii! . 
&compuced_line [i 1 ) ; 
avg_squared__error += f it_errar.' 4 ; 
img^cingie [i] = comDuted_line (ii . c ; 

} 

♦f eature_acare » 

cz_map_average_squared_error_co_max_1000 (avg^squared error* ; 

/* Average Che four rays by averaging the positions and 
* oriencacions 
*/ 

compuced_line [01 .x 
compuced_line [0] .y 
computed_line [1] .x 
compur sd_ I me [ 1 J . y 

angle_horiz line = 

average_f our_image_orienrat:icns '. calib , i:r.c ar.r.e 
compuced_line [0] .t = angle horiz line; 

compuned_iine [i; . r = image_ncrT.al_io__image_angle . zalib, ancle hcriz line: 

-wa_lines_ro_poinc ( &comput:ed_line [0] , ^computed line [I] . conipured cen-er • ; 

•compuced_orientat:ion_phys = 

transf crn_angle_according_uC_calinraricn map 

( ( (priv_cwa_calib ♦) calib? - >nao . ancle hor'2 i'r'=*-- 

/* cwa_f ind_target: 0 is the rop level Suncricn for localizing 

* the crosshair target. It returns C if it cannot find the crosshair. 

* 1) use search to improve position estimate of crosshair center 

* 2) compute four good rays eminatmg from the origin by sampling 

* edge points within the guaranteed good region along the user 

* specified orientacion 

* 3) Enumerate subpixel edge points along the four rays eminating 

* from the the origin and use the found good ravs to identify outliers 

* cwa_f ind_target tries to localize a crosshair target within an 

* image and returns 1 if it was successful and 0 otherwise 

* img - cip_buffer containing crosshair target 

* calib - characterizing field of view 

* angle - initial orientation estimate of crosshair taraet 

* result - pointer to cwa_line structure where result will be stored 
*/ 

inc cwa_f ind_target_intemal 

(cip_buffer -img, cwa_calib ♦calib, double angle, cwa_line. *result. 
cwa_3tatus * status) 
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= (computed_line [0] .X + ccmputec_line [; ] . x; ., 2 

= (computed_line [ 0) , y + comoutec line;2].y^ 2 

= (computed_line i 1] . X comouted line [ 3 ] . x; ; 2 

= ( computed^l me [ 1 ] y *- ccxc-tec line '3] . '2 
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cwa_point: gro9S_pos_escimace , irnproved_pos_-2srimare ; 
double improved jhys_or lent a cion^e SCI mace 
cwa_line good_ray3 [4 1 ; 
inc x.ana: 
ccc_signal sig; 
ccm_cimer citne; 
inc C0.tl,t2,c3; 
cwa_Iine cmp; 

cwa_calib •calib_idencicy=NULL; 
NO_REGISTER(calib_idencicy) ; 
cu_clear (scacus , sizeof (cwa_scacus) ) ; 

/♦ CWA_ERR_NO_INTERSECTION is used Co signify when we car^-.cc fini 
* any edge points for an expecced horircncal or vercical line 
•/ 



if ,*sig = ccc_cacch i 0 i . 
gcco done ; 



e;cpand che scack cc rnake rccT. frr 
allocace 



cu_expand_scack ; 1 S ) 

if Cresulc) 
result = icxp; 



if (timg) 

ccc_errcr •;CGEN_ERH_3ADAHG'- ; 

/* If no calibration object is passed, T.ake ^ne so chac chere is 

* only one pach through the code 

♦ / 

if Ccalib) 

calib = calib_idencity = cwa_calib_mak.e :o.,0.,0.,l..l.,0); 
/♦ 1) use search */ 



ccm_begin ( &time) ; 

ans = gross_locace_crosshair_center_in_image 

(img, (int) (10. * angle* 190/?! : , igross_pcs_estir\ace . ; 
if (:ans) cct error (CWA_ERR_NC_ INTERSECT ) ; 



cip_buffer win; 
double x_edge, y_edge; 
inc edge_val , dir ; 
double angle_of f sec ; 
ccc_3ignal sig; 
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inc 1 ; 

double lef t [4] ={ -2 . . -2 . , 2 . , 2 . / ; 
double top 1 4 ] = ( - 2 . , 2 . , 2 . . - 2 . ; 
cip_window ( img , iwin , 

gross_pos__escirnace . x - 3 5 , gross_pcs esr.iia -e . v - 3 5 , , - : 
for (i = 0 ; i < 4 ; i^-) ( 

if (c2_cebc_starc_recum_success_vai 

(&win, 3,3, lef t [i] , cop [i] , i;c_edge, iy_edge, iedge^val. 
idir, &angle_of f sec) ) 
break ; 

) 

scatus- >concrasc_score=cz_map_edge_val_to_max 1000 {'»dae val' ■ 

} " " " " 

to = ctm_read ( ttime) ; 

if ( cwa_£ind_target_debug_f lag i 0x1; ( 
princf ( "gross pos : " ) ; 

cd_showas i (ir*c) igross_pcs_es^i:naze . "cwa_ccir.c" ■ 

} - - . 

/♦ 2) improve che pcsic-cn and orier.cacicr. esti~ace and ccrrpuce 
* Che four good rays 



compuce_cencer_pos:.Cicn_ar.d_a"gle_as3umng_f cur_gi:arar.ceed rc-d ravs 
(img, calib, igross_cos_escmace . 
cransf orm_angle_acccrd-ng_cc_ralibrac:.or._map 
( ( (priv_cwa_calib •) calib- - >map, angle; . 
(doxifale) GUAaAKTZED_GCCD_SIZc:, iir.proved_t:cs_escLr:iace , 
&improved_phys_orier.caticr._esciniace , cccd_rays ) ; 

tl = ccnr^read ; icimei ; 

if { cwa_f ind_carget_debug_f lag i 0x1) ( 
princf ' " improved_pos_escimace : " ) ; 

cd_showas ( ;inc) &irTiDroved_pcs_escimace , "cwa_pcinc'*) ; 

} 

/♦ 3) Enumerace edge pomes along chose -four good rays and use 
♦ those lines co compute the final position and orientation estiT.aces 
*/ 

contpute_center_posicion_aLnd_angle_along_guaranteed_good rays 

(img, calib, fitimproved_pos_estimace , gcod_rays , TIGHT THRESHOLD, 
(cwa_point *) result , i ( result - >t) , 
istacus- >f eacure__score) ; 

t2 = ctm_read ( ttime) ; 

c 2_c rans f ormPo int 

i { {priv_cwa_calib *j calib; - >map. 
result->x, resulc->y. 
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ireault - >x. iresuic - >y) ; 

if (cwa_f ind_cargec_debug__f lag i 0x1) ( 
princf result : ") ; 

cd_3howas ( { inc) result , "cwa_pc:.nc" * ; 

} 

if (cwa_find_cargec_debug_f lag ^ 0x4 ) ( 
printf C^Times : ") ; 

princfC gross locate: Vd\n'\ tO) ; 
printf (" fine locate: Vd\n" , tl - tO ) ; 
printf (" finer locate: %d\n", t2 - tl ) ; 

) 

done : 

cu__expand_stack ( - 16) ; 

if (calib_identity) cwa_calib_delete •;calib_identiiy: , 
/♦ Make returned angle agree wirh given angle argument 

if (sig) cct_throwtsig) ; 

for (i = -8; i <= 3; i--i 
if ( (i ! = 0) 

( fabs (result- >t - i*PI. 2 - angle; < fabs .resuli - >r - angle 
result ->t i*PI,2 , i--; 

return 1 ; 

) 

int cwa_f ind_target 

(cip_buffer 'img, cwa^calib ♦calic, double angle. cwa_lir.e -icsult, 
cwa_status •status) 

{ 

cwa_status tmp_status; 
cct_signal sig; 

if (1 status) status = &tmp_status; 

cct_catch{CGEN_ERR_BADARG) ; 
cct_catch(C:WA_ERR_BACiaiT) ; 
cct_catch{CWA_ERR_TRACKZR) ; 

if ( s ig=cct_catch ( 0 ) ) { 

if (sig == CGEN_ERR_BADARG) { 

status- >error_f lags i= O^A_ERR_3ADARG ; 
cct_end (sig) ; 
cct_error (sig) ; 

} 

status- >error_f lags | = O^A_ERR_r AILZD_TC_FIND_TARGET; 
retum status- >found = 0; 

I 

cwa_f ind_target_intemal ( img, calib. angle , result , status ) ; 
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return status -> found =1; 

} 
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Described herein are calibration targets and methods for determining the location 
and orientation thereof meeting the objects set forth above. It will be appreciated that the 
embodiments shown in the drawings and discussed above are merely illustrative of the 
invention, and that other embodiments incorporating changes therein fall within the scope 
of the invention, of which we claim: 
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1. 



10 



15 

2. 



20 



A machine vision method of deterniining a location of a reference point in an 
image of a calibration target, the method comprising the steps of 



A. generating an image of a target that includes 

i- two or more regions, each region being defined by at least two linear 
edges that are directed toward a reference point; 

ii- at least one of the regions having a different imageable characteristic 
from an adjacent region; 

B. identifying edges in the image corresponding to lines in the calibration target; 

C. determining a location where lines fitted to those edge ■ 



^es mtersect. 



A method according to step 1. wherein step (B) comprises the step of identifying 
edges in the image by applying an edge detection vision tool to the image. 

3. A method according to step 1, wherein step (B) comprises identifying edges in the 
image by applying a caliper vision tool to the image. 

4. A method according to step 1, wherein step (B) comprises identifying edges in the 
image by applying a first difference operator vision tool to the image, and by 
applying a peak detector vision tool the an output of the first difference operator 

25 vision tool 

5. A method according to step 1, wherein step (B) comprises identifying edges in the 
miage by applying a caliper tool to the image beginning at an approximate location 

of the reference point. 

30 
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6. A method according to claim 5, comprising the step of determining an approximate 
location of the reference point by applying a Hough line vision tool to the image 
and finding an intersection of lines identified thereby. 



5 7. 



10 8. 



15 



A method according to claim 5, comprising the step of determining an approximate 
location of the reference point by applying a correlation vision tool to the image 
using a template substantially approximating an expected pattern of edges in the 
image. 

A method according to claim 5, comprising the step of determining an approximate 
location of the reference point by applying a projection vision tool to the image 
along one or more axes with which the edges are expected to be aligned. 



9. A method according to claim 1, comprising the steps of 



identifying edges in the image corresponding to lines in the calibration target; and 



determining an orientation of the target based on the angle of those identified 
edges. 

20 

10. A machine vision according to claim 1, wherein 

step (B) includes the step of applying a Hough line vision tool to identify edges 
the image corresponding to lines in the calibration target; and 



25 



30 



m 



the method further including the step of determining an orientation of the target 
based on the angle of those identified edges. 

11. A machine vision according to claim 1 , comprising 

A. applying a Sobel edge tool to the image to generate at least a Sobel angle image; 
and 
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B. taking an angle histogram of the Sobel angle image to determine an orientation of 
the target. 



12. A machine vision method of an orientation of a calibration target in an image, the 
method comprising the steps of 

A. generating an image of a target that includes 

1. two or more regions, each region being defined by at least two linear 

edges that are directed toward a position; 

ii- at least one of the regions having a different imageable characteristic 
fi"om an adjacent region; 



15 B. identifying edges in the image corresponding to lines in the calibration 



target; 



C. determining an orientation of the target based on the angle of those identified 
edges. 

20 13. A method according to claim 12, wherein 

step (B) includes applying a Hough line vision tool to the image to identify therein 
edges in the image corresponding to lines in the caUbration target; and 

25 step (C) includes determining an orientation of the target based on the angle of 

those identified edges. 

14. A method according to claim 12, wherein 



30 



step (B) includes applying a Sobel edge tool to generate at least a Sobel angle 
image; and 
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Step (C) includes taking an angle histogram of the Sobel angle image to determine 
an orientation of the target. 



A method according to any of claims 1 and 12. wherein step (A) includes the step 
of generating an image of a target in which each region has any of a diflferent 
color, contrast, brightness and stippling from regions adjacent thereto. 

A method according to claim 15, for use with a machine vision system having 
image capture means for capturing an image of the target, wherein each region has 
a different characteristic in such an image than the regions adjacent thereto. 

A method according to any of claims 1 and 12, wherein step (A) includes the step 
of generating an image of a urget for which the reference point is at a center 
thereof 

A method according to claim 17, wherein step (A) includes the step of generating 
an image of a target for which the linear edges substantially meet at the reference 
point. 

A method according to any of claim 1 and 12, wherein step (A) includes the step of 
generating in image of a target ha\dng four imageable regions. 

A method according to claim 19, wherein step (A) includes the step of generating 
an image of a four-region target in which the at least two linear edges of each of 
the regions are perpendicular to one another. 

A machine vision method of determining a location and orientation of a calibration 
target in an image thereof, the method comprising the steps of 

generating an image of a target that includes 
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1. two or more regions, each region being defined by at least two linear 

edges that are directed toward a reference point; 

li.. at least one of the regions having a different imageable characteristic 
fi'om an adjacent region; 

B. generating, based on analysis of the image, an estimate of an orientation of the 
target therein; 



10 C. 



15 



20 



25 



generating, based on analysis of the image, an estimate of a location of the 
reference point therein; 



D. refining, based on analysis of the image, estimates of at least one of the location of 
the reference point in the image and the orientation of the target in the image. 

22. A method according to claim 21, wherein step (B) includes one or more of the 
following steps: 

i- applying a Hough line vision tool to the image to find an angle of edges 

in the image; 

ii. applying a Sobel edge tool to the image to generate Sobel angle image, 
generating a histogram of the angles represented in that Sobel angle 
image, and determining a one dimensional correlation thereof; and 

lu. inputting an orientation from the user. 

23. A method according to claim 22, wherem step (C) includes one or more of the 
following steps: 
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i. applying a Hough vision tool to the image to find an approximate 

location of edges therein, and finding an intersection of lines defined by 
those edges; 

5 ii. applying a projection tool vision tool to the image to find an approximate 

location of edges therein, and finding an intersection of lines defined by 
those edges; 

iii. applying a correlation vision tool to the image to find approximate 
10 coordinates of the reference point; and 

iv. determining a sum of absolute differences to find approximate 
coordinates of the reference point. 



15 24, 



25 25. 



A method according to claim 23, wherein step (D) includes the step of invoking 
the steps in sequence, at least one time: 



i. applying a caliper vision tool along each of the expected edges in the 
image, where those edges are defined by a current estimate of the 
reference point location and a current estimate of the target orientation; 
and 

ii. fitting lines to edge points determined by the caliper vision tool. 



30 



A method according to claim 24, wherein step (D)(i) includes the step of applying 
the caliper vision tool along each of the expected edges until any of the following 
conditions is met: no edge is found by the caliper vision tool at a sample point; 
more than one edge is found by the caliper vision tool at a sample point; a distance 
between a sample edge point and a nominal line is larger than a threshold; or, the 
caliper vision tool window for the sample edge point extends outside of the image. 
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